Participatory monitoring and evaluation approaches that influence decision-making: lessons from a maternal and newborn study in Eastern Uganda
Tóm tắt
The use of participatory monitoring and evaluation (M&E) approaches is important for guiding local decision-making, promoting the implementation of effective interventions and addressing emerging issues in the course of implementation. In this article, we explore how participatory M&E approaches helped to identify key design and implementation issues and how they influenced stakeholders’ decision-making in eastern Uganda. The data for this paper is drawn from a retrospective reflection of various M&E approaches used in a maternal and newborn health project that was implemented in three districts in eastern Uganda. The methods included qualitative and quantitative M&E techniques such as key informant interviews, formal surveys and supportive supervision, as well as participatory approaches, notably participatory impact pathway analysis. At the design stage, the M&E approaches were useful for identifying key local problems and feasible local solutions and informing the activities that were subsequently implemented. During the implementation phase, the M&E approaches provided evidence that informed decision-making and helped identify emerging issues, such as weak implementation by some village health teams, health facility constraints such as poor use of standard guidelines, lack of placenta disposal pits, inadequate fuel for the ambulance at some facilities, and poor care for low birth weight infants. Sharing this information with key stakeholders prompted them to take appropriate actions. For example, the sub-county leadership constructed placenta disposal pits, the district health officer provided fuel for ambulances, and health workers received refresher training and mentorship on how to care for newborns. Diverse sources of information and perspectives can help researchers and decision-makers understand and adapt evidence to contexts for more effective interventions. Supporting districts to have crosscutting, routine information generating and sharing platforms that bring together stakeholders from different sectors is therefore crucial for the successful implementation of complex development interventions.
Tài liệu tham khảo
World Health Organization. Monitoring Maternal, Newborn and Child Health: Understanding Key Progress Indicators. Geneva: WHO; 2011. http://apps.who.int/iris/bitstream/10665/44770/1/9789241502818_eng.pdf. Accessed 12 Dec 2017.
Maine D, Akalin MZ, Ward VM, Kamara A. The Design and Evaluation of Maternal Mortality Programs. New York: Center for Population and Family Health, Columbia University; 1997. https://www.mailman.columbia.edu/sites/default/files/pdf/designevalmm-en.pdf. Accessed 12 Dec 2017.
Holvoet N, Inberg L. Taking stock of monitoring and evaluation systems in the health sector: Findings from Rwanda and Uganda. Health Policy Plan. 2014;29:506–16. doi:10.1093/heapol/czt038.
Bell JS, Marais D. Participatory training in monitoring and evaluation for maternal and newborn health programmes. Glob J Health Sci. 2015;7:192–202. doi:10.5539/gjhs.v7n2p192.
Baum F, Lawless A, Delany T, Macdougall C, Williams C, Broderick D, et al. Evaluation of Health in All Policies: concept, theory and application. Health Promot Int. 2014;29 Suppl 1:i130–42. doi:10.1093/heapro/dau032.
Reynolds HW, Sutherland EG. A systematic approach to the planning, implementation, monitoring, and evaluation of integrated health services. BMC Health Serv Res. 2013;13:168. doi:10.1186/1472-6963-13-168.
MEASURE Evaluation. Safe Motherhood — MEASURE Evaluation. http://www.cpc.unc.edu/measure/prh/rh_indicators/specific/sm. Accessed 27 Sep 2016.
Darmstadt GL, Marchant T, Claeson M, Brown W, Morris S, Donnay F, et al. A strategy for reducing maternal and newborn deaths by 2015 and beyond. BMC Pregnancy Childbirth. 2013;13:216. doi:10.1186/1471-2393-13-216.
Hernandez M, Hodges S. Applying a theory of change approach to interagency planning in child mental health. Am J Community Psychol. 2006;38:165–73. doi:10.1007/s10464-006-9079-7.
Cousins JB, Earl LM. The Case for Participatory Evaluation: Theory, Research, Practice. London: Falmer; 1995.
Weaver L, Cousins JB. Unpacking the participatory process. J Multidiscip Eval. 2004;1:19–40.
Estrella M, Gaventa J. Who Counts Reality? Participatory Monitoring and Evaluation: A Literature Review. Economic development projects. Brighton: Institute of Development Studies at the University of Sussex; 1998. doi:10.1017/CBO9781107415324.004.
Rath S, Nair N, Tripathy PK, Barnett S, Rath S, Mahapatra R, et al. Explaining the impact of a women’s group led community mobilisation intervention on maternal and newborn health outcomes: the Ekjut trial process evaluation. BMC Int Health Hum Rights. 2010;10:25. doi:10.1186/1472-698X-10-25.
Rowlands G, Sims J, Kerry S. A lesson learnt: the importance of modelling in randomized controlled trials for complex interventions in primary care. Fam Pract. 2005;22:132–9. doi:10.1093/fampra/cmh704.
Kusters K, Buck L, de Graaf M, Minang P, van Oosten C, Zagt R. Participatory planning, monitoring and evaluation of multi-stakeholder platforms in integrated landscape initiatives. Environ Manage. 2017. doi:10.1007/s00267-017-0847-y.
Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321:694–6. doi:10.1136/bmj.321.7262.694.
Baker JL. Directions in Development: Evaluating the Impact of Development Projects on Poverty A Handbook for Practitioners. Washington, DC: The World Bank; 2000. http://siteresources.worldbank.org/INTISPMA/Resources/handbook.pdf. Accessed 12 Dec 2017.
Smits PA, Champagne F. An assessment of the theoretical underpinnings of practical participatory evaluation. Am J Eval. 2008;29:427–42.
Cousins JB, Chouinard JA. Evaluation and Society: Participatory Evaluation up Close: An Integration of Research-Based Knowledge. Greenwich: Information Age Publishing, Inc.; 2012.
Ekirapa-kiracho E, Tetui M, Bua J, Muhumuza R, Waiswa P, Makumbi F, et al. Maternal and neonatal implementation for equitable systems. A study design paper. Glob Health Action. 2017;10(sup4):1346925. doi:10.1080/16549716.2017.1346925.
Mark MM. Evaluation’s future: furor, futile, or fertile? Am J Eval. 2001;22:457–79.
Uganda Bureau of Statistics. 2014 Statistical Abstract. Kampala: UBOS; 2014.
Uganda Bureau of Statistics (UBOS) and ICF International Inc. Uganda Demographic and Health Survey 2011. Kampala: UBOS and Calverton: ICF International Inc.; 2012. https://dhsprogram.com/pubs/pdf/FR264/FR264.pdf. Accessed 12 Dec 2017.
Uganda Bureau of Statistics (UBOS) and ICF International Inc. Uganda Demographic and Health Survey 2016: Key Indicator Survey. Kampala: UBOS and Calverton: ICF International Inc.; 2017 https://dhsprogram.com/pubs/pdf/PR80/PR80.pdf. Accessed 12 Dec 2017.
Kananura RM, Tetui M, Mutebi A, Bua JN, Waiswa P, Kiwanuka SN, et al. The neonatal mortality and its determinants in rural communities of Eastern Uganda. Reprod Health. 2016;13:13. doi:10.1186/s12978-016-0119-y.
Pain L, Wilkinson A, Tetui M, Ekirapa-Kiracho E, Barman D, Ahmed T, et al. Using Theories of Change to inform implementation of health systems research and innovation: experiences of Future Health Systems consortium partners in Bangladesh, India and Uganda. Health Res Policy Syst. (12961-15-S2-S4 in this issue)
Davies R, Dart J. The “Most Significant Change” (MSC) Technique. 2005. http://www.mande.co.uk/docs/MSCGuide.pdf. Accessed 12 Dec 2017.
Ekirapa-Kiracho E, Ghosh U, Brahmachari R, Paina L. Engaging stakeholders: lessons from the use of participatory tools for improving maternal and child care health services. Health Res Policy Syst. (12961-15-S2-S3 in this issue)
Kisakye AN, Kananura RM, Ekirapa-Kiracho E, Bua J, Akulume M, Namazzi G, et al. Effect of support supervision on maternal and newborn health services and practices in Rural Eastern Uganda Effect of support supervision on maternal and newborn health services and practices in Rural Eastern Uganda. Glob Health Action. 2017;10(sup4):1345496. doi:10.1080/16549716.2017.1345496.
Moore GF, Audrey S, Barker M, Bonell C, Hardeman W, Moore L, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258. doi:10.1136/bmj.h1258.
de Vlaming R, Haveman-Nies A, Van’t Veer P, de Groot LC. Evaluation design for a complex intervention program targeting loneliness in non-institutionalized elderly Dutch people. BMC Public Health. 2010;10:552. doi:10.1186/1471-2458-10-552.
Chaillet N, Dubé E, Dugas M, Francoeur D, Dubé J, Gagnon S, et al. Identifying barriers and facilitators towards implementing guidelines to reduce caesarean section rates in Quebec. Bull World Health Organ. 2007;85:791–7. doi:10.2471/BLT.06.039289.
Evans-Lacko S, Jarrett M, McCrone P, Thornicroft G. Facilitators and barriers to implementing clinical care pathways. BMC Health Serv Res. 2010;10:182. doi:10.1186/1472-6963-10-182.
Camden C, Swaine B, Tétreault S, Carrière M. Going beyond the identification of change facilitators to effectively implement a new model of services: lessons learned from a case example in paediatric rehabilitation. Dev Neurorehabil. 2011;14:247–60. doi:10.3109/17518423.2011.577049.
Addington D, Kyle T, Soni D, Wang J. Facilitators and barriers to implementing quality measurement in primary mental health care: systematic review. Can Fam Physician. 2010;56:1322–31.
Schultz JA, Pandya S, Sims M, Jones JA, Fischer S. Participatory monitoring and evaluation within a statewide support system to prevent adolescent substance abuse. J Prev Interv Community. 2013;41:188–200. doi:10.1080/10852352.2013.788347.
Braithwaite RL, McKenzie RD, Pruitt V, Holden KB, Katrina Aaron M, Chavone Hollimon M. Community-based participatory evaluation: the health start approach. Health Promot Pract. 2013;10:54–6. doi:10.1177/1524839912443241.
Muriisa RK. Decentralisation in Uganda: prospects for improved service delivery. Africa Dev. 2008;XXXIII:83–95.
Bashaasha B, Mangheni MN, Nkonya E. Decentralization and Rural Service Delivery in Uganda. Washington, DC: IFPRI; 2011.
Hotchkiss D, Diana M, Foreit K. How Can Routine Health Information System Improve Health System Function in Low-Resource Setting. 2012. https://www.measureevaluation.org/resources/publications/sr-11-65. Accessed 17 Oct 2016.
Kothari A, Rudman D, Dobbins M, Rouse M, Sibbald S, Edwards N. The use of tacit and explicit knowledge in public health: a qualitative study. Implement Sci. 2012;7:20.