Context to the influence of evaluations: An essay on meta-evaluation with usefulness and accountability enfasis
Abstract
Objective: to systematize the meta-evaluation methodology in its conceptual and operational aspects to facilitate its use in health management practices.
Methods: this is an essay on meta-evaluation. The theoretical bases were searched in books and articles, published between 1991 and 2018, that approached the conceptual, methodological and operational aspects of the meta-evaluation.
Results: the meta-evaluation consists of a process of delineating, obtaining and applying descriptive information and judgment about the usefulness, feasibility, adequacy and precision of an evaluation in its systematic nature, based on standards and criteria proposed and validated. There are several ways or approaches to the meta-evaluation. However, the standards commonly follow the guidelines of the Joint Committee on Standards for Educational Evaluation, which proposes 30 standards grouped into five principles: utility, viability, ownership, accuracy / precision and accountability, being these essential and determinants for the quality of an evaluative study.
Conclusion: the meta-evaluation contributes to the judgment of the quality of the evaluation processes, in the adequacy and usefulness, and can be useful, especially when it comes to themes and contexts that have already been evaluated and therefore is also described as "an evaluation of an evaluation".
Downloads
References
Hartz ZMA, Contandriopoulos AP. Integralidade da atenção e integração de serviços de saúde: desafios para avaliar a implantação de um “sistema sem muros”. Cad. Saúde Pública. 2004; 20(Suppl 2): S331-S336.
Machado TR, Chaise RM, Elliot LG. A meta-avaliação como instrumento de qualidade nas Políticas Públicas: o Programa Segundo Tempo. Rev Meta Aval, 2016; 8: 1-20.
Davidson EJ. Evaluation methodology basics. Thousands Oaks, CA: Sage, 2005.
Elliot LG. Metaevaluation: from approaches to possibilities of application. Ensaio: Avaliação e Políticas Públicas em Educação, 2011; 19(73): 941-64,.
Hartz Z. Meta-avaliação da atenção básica à saúde – teoria e prática. Rio de Janeiro: FIOCRUZ, 2008, 410p.
Stufflebeam DL. The metaevaluation imperative. American Journal of Evaluation, 2001; 22(2): 183-209.
Hartz ZMA. Principles and standards in metaevaluation: guidelines for health programs. Ciência & Saúde Coletiva, 2006; 11(3): 733-38.
Guerreiro CS, Ferrinho P, Hartz ZMA. Avaliação em saúde na República da Guiné-Bissau: uma meta-avaliação do Plano Nacional de Desenvolvimento Sanitário. Saúde em Debate, 2018; 42: 549-65.
Silveira R. Um exame da organização textual de ensaios científicos. Revista Estudos Linguísticos, 1992; 21: 1244-52.
Joint Committee on Standards for Educational Evaluation. The program evaluation standards: a guide for evaluators and evaluation users: a guide for evaluators and evaluation users. 3rd. ed. Thousands Oaks, CA: Sage, 2011.
Fitzpatrick JL, Sanders JR., Worthen BR. Program Evaluation: Alternative Approaches and Practical Guidelines, 4th ed. Boston: Pearson Education, 2011.
Cooksy LJ, Caracelli VJ. Meta-evaluation in Practice. Journal of multidisciplinary evaluation, 2009; 6(11): 1-15.
Hartz ZMA. Meta-avaliação da gestão em saúde: desafios para uma” nova saúde pública”. Ciência & Saúde Coletiva, 2012; 17: 832-34.
Scriven M. Evaluation thesaurus. 4th. ed. Newbury Park, CA: Sage, 1991.
Khalil H. et al. An evidence – based approach to scoping reviews. Worldviews on Evidence – Based Nursing, 2016; 13(2): 118-23.
Figueiró AC, Hartz ZMA, Thuler LCS, Dias ALF. Meta-evaluation of baseline studies of the Brazilian Family Health Strategy Expansion Project: a participatory and formative approach. Cad. Saúde Pública. 2010; 26(12): 2270-78.
Scott-Little C, Hamann MS, Jurs SG. Evaluations of after-school programs: A meta-evaluation of methodologies and narrative synthesis of findings. American Journal of Evaluation, 2020; 23(4): 387-419.
Joint Committee on Standards for Educational Evaluation. The program evaluation standards: how to assess evaluation of educational programs. Hightstown, NJ: McGraw-Hill Book Co., 1981.
Joint Committee on Standards for Educational Evaluation. The program evaluation standards: how to assess evaluation of educational programs. 2nd. ed. Thousands Oaks, CA: Sage, 1994.
Joint Committee on Standards for Educational Evaluation. Checklist of The Program Evaluation Standards Statements. Western Michigan University, 2018.
Hartz ZMA et al. Multi-strategy in the Evaluation of Health Promotion Community Interventions: An Indicator of Quality. In: Potvin L, Mcqueen D. (eds.), Health Promotion Evaluation Practices in the Americas, DOI: 10.1007/978-0-387-
-5 14, C Springer Science Business Media, LLC, 2008.
Bardin L. Análise de conteúdo. São Paulo: Edições 70. 2011.
Scriven M. Evaluation thesaurus. 4th. ed. Newbury Park, CA: Sage, 1991.
Stufflebeam DL. Program Evaluations Summary Metaevaluation Checklist (Based on The Program Evaluation Standards). 2012.
Fronteira I, Ferrinho F, Dussault G, et al. Relatório de Avaliação Final da Execução do Plano Nacional de Desenvolvimento Sanitário 2003-2007 da República da Guiné-Bissau. Relatório de Avaliação Final, Associação para o Desenvolvimento e Cooperação Garcia de Orta, 2007.
Brasil, Ministério da Saúde. Pública da República da Guiné Bissau. Plano Nacional de Desenvolvimento Sanitário 1998-2002. 1997.
Brasil, Ministério da Saúde Pública da República da Guiné-Bissau. Plano Nacional de Desenvolvimento Sanitário II 2008-2017. 2007.
Hartz ZMA. Meta-evaluation of health management: challenges for ‘new public health’. Ciênc Amp Saúde Coletiva 2012; 17: 832–834.
Isabella Samico, Eronildo Felisberto, Ana Claudia Figueiró, et al. Avaliação em Saúde: bases conceituais e operacionais. Medbook, Editora Científica. Rio de Janeiro, 2010.
Furtado JP, Laperrière H. Parâmetros e paradigmas em meta-avaliação: uma revisão exploratória e reflexiva. Ciência & Saúde Coletiva, 2012; 17: 695-705.
Cotera AT, Matamoros AA. Propuesta para implementar la Metaevaluación en la Escuela de Ciencias de la Educación de la Universidad Estatal. Revista Calidad en la Educación Superior, 2011; 2(1): 247-65.
Worthen BR, Sanders JR, Fitzpatrick JL. Avaliação de programas: concepções e práticas. São Paulo: Ed. Gente, 2004.
Vieira LMS, Souza LEPF, Hartz ZMA. Conferência de consenso sobre a imagem-objetivo da descentralização da atenção à saúde no Brasil. In: Vieira LMS, Hartz ZMA. (Org.). Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. 1ed. Rio de Janeiro; Salvador: FIOCRUZ/EDUFBA, 2005, v. , p. 65-102.
Dias Sobrinho J. Avaliação: políticas educacionais e reformas da educação superior. São Paulo: Cortez, 2003.
This work is licensed under a Creative Commons Attribution 4.0 International License.
