Applying the results based management framework to the CERCA multi-component project in adolescent sexual and reproductive health: a retrospective analysis
Tóm tắt
Adolescent sexual and reproductive health (SRH), problems such as unplanned pregnancies are complex and multifactorial, thus requiring multifaceted prevention interventions. Evaluating the impact of such interventions is important to ensure efficiency, effectiveness and accountability for project funders and community members. In this study, we propose Results Based Management (RBM) as a framework for project management, using the Community Embedded Reproductive Health Care for Adolescents (CERCA) as a case study for RBM. The CERCA Project (2010-2014) tested interventions to reduce adolescent pregnancy in three Latin American countries, Bolivia, Ecuador and Nicaragua. Activities were designed to increase adolescent SRH behaviors in four domains: communication with parents, partners and peers; access to SRH information; access to SRH services; and use of contraception. When the project ended, the outcome evaluation showed limited impact with concerns about accuracy of monitoring and attrition of participants. We reviewed and analyzed a series of CERCA documents and related data sources. Key findings from these documents were organized within an RBM framework (planning, monitoring, and impact evaluation) to understand how CERCA methodology and performance might have reaped improved results. Strengths and weaknesses were identified in all three elements of the RBM framework. In Planning, the proposed Theory of Change (ToC) differed from that which was carried out in the intervention package. Each country implemented a different intervention package without articulated assumptions on how the activities of intervention would bring about change. In Monitoring, the project oversight was mainly based on administrative and financial requirements rather than monitoring fidelity and quality of intervention activities. In Impact Evaluation, the original CERCA evaluation assessed intervention effects among adolescents, without identifying success and failure factors related to the outcomes, the nature of the outcomes, or cost-effectiveness of interventions. This analysis showed that multi-country projects are complex, entail risks in execution and require robust project management. RBM can be a useful tool to ensure a systematic approach at different phases within a multi-country setting.
Tài liệu tham khảo
Bester A. Results-based Management in the United Nations Development System: progress and challenges. In: A report prepared for the United Nations Department of economic and social affairs, for the quadrennial comprehensive policy review. New York: United Nations; 2012.
Busjeet G. Planning, monitoring, and evaluation : methods and tools for poverty and inequality reduction programs. Washington, DC: World Bank; 2013.
Cordova, K. & Hagens, A., 2013. Information and Communications Technology in Sexual and reproductive health care for Adolescents - A Bolivian Case Study. GSTF, pp.112-117. ISSN: 2251-3833. https://doi.org/10.5176/2251-3833_GHC13.59.
Cordova-Pozo K, et al. Improving adolescent sexual and reproductive health in Latin America: reflections from an international congress. Reprod Health. 2015;12(12):11.
Crescendo Stat-Gent. The CERCA-project: community embedded reproductive health interventions for adolescents in Latin America. Ghent: ICRH, Ghent University; 2014.
de Meyer S, et al. A cross-sectional study on attitudes toward gender equality, sexual behavior, positive sexual experiences, and communication about sex among sexually active and non-sexually active adolescents in Bolivia and Ecuador. Glob Health Action. 2014;7:24089. https://doi.org/10.3402/gha.v7.24089.
Decat, P. et al., 2015. Sexual onset and contraceptive use among adolescents from poor neighbourhoods in Managua, Nicaragua. Eur J Contracept Reprod Health Care., Mar 4; 20(2)(doi: https://doi.org/10.3109/13625187.2014.955846), p. 88–100.
Decat P, et al. Community embedded reproductive health interventions for adolescents in Latin America: development and evaluation of a complex multi-centre intervention. BMC Public Health, Issue 2013 13:31. 2013;13:31.
Denno, D., Hoopes, A. & Chandra-Mouli, V., 2014. Effective strategies to provide adolescent sexual and reproductive health services and to increase demand and community support.
Echeverri, M. What we should know about Community-Based Participatory Research. 2010. [Accessed 2017 Abr 10]. Available from: https://lacats.pbrc.edu/documents/About%20CBPR%2010-08-13.pdf.
Hainsworth G, Simon D, Rahimtoola M, Ghiron L. Scale-up of adolescent contraceptive services: lessons from a 5-country comparative analysis. J Acquir Immune Defic Syndr. 2014;66(Supplement 2):S200.
HIVOS, 2012. How can ToC thinking help in monitoring and evaluation?. [Online] Available at: http://www.hivos.net/Hivos-Knowledge-Programme/Themes/Theory-of-Change/Resources/4.-How-can-ToC-thinking-help-in-monitoring-and-evaluation [Accessed 21 July 2015].
Ivanova O, et al. Lessons learnt from the CERCA project, a multicomponent intervention to promote adolescent sexual and reproductive health in three Latin America countries: a qualitative post-hoc evaluation. Eval Program Plann. 2016;58:98–105. (DOI: 10.1016/j.evalprogplan.2016.06.007), pp. 98-105
Jaruseviciene L, et al. Primary healthcare providers’ views on improving sexual and reproductive healthcare for adolescents in Bolivia, Ecuador, and Nicaragua. Glob Health Action. 2013;6:20444. https://doi.org/10.3402/gha.v6i0.20444.
Leeuw, F., & Donaldson S., 2015. Theory in evaluation: Reducing confusion and encouraging debate. Sage Journals. Vol. 21, issue 4, pp. 467-480. https://doi.org/10.1177/1356389015607712.
Michielsen K, et al. Effectiveness of complex interventions: post hoc examination of the design, implementation and evaluation of the CERCA project to understand the results it achieved. Ghent: ICRH, Ghent University; 2015.
Nelson E, et al. The unintended consequences of sex education: an ethnography of a development intervention in Latin America. Anthropology & Medicine. 2014;21(2):189–201. https://doi.org/10.1080/13648470.2014.918932.
Njeuhmeli, E., Hatzold, K. & Gold, E., et al., 2014. Lessons Learned from Scale-Up of Voluntary Medical Male Circumcision Focusing on Adolescents: Benefits, Challenges, and Potential Opportunities for Linkages with Adolescent HIV, Sexual, and Reproductive Health Services. J Acquir Immune Defic Syndr. 2014 Jul 1;Vol. 66 Suppl 2. Pp.193-199. https://doi.org/10.1097/QAI.0000000000000179.
Oringanje, C., Meremikwu, M., Eko, H. & Esu, E., 2016. Interventions for preventing unintended pregnancy among adolescents. Cochrane Fertility Regulation Group, Feb 3;2:CD005215. https://doi.org/10.1002/14651858.CD005215.pub3.
Rahmani, R., 2012. Donors, beneficiaries, or NGOs: whose needs come first? A dilemma in Afghanistan. Journal Development in Practice, Vol. 22, Iss. 3,2012. Pp.295-304. https://doi.org/10.1080/09614524.2012.664622.
Rogers P. Overview of impact evaluation, methodological briefs: impact evaluation 1. Florence: UNICEF Office of Research; 2014.
Schunter, B., Cheng, W., Kendall, M. & Marais, H., 2014. Lessons Learned From a Review of Interventions for Adolescent and Young Key Populations in Asia Pacific and Opportunities for Programming. JAIDS Journal of Acquired Immune Deficiency Syndromes: July 1st, 2014. Vol.66, Issue 2, pp.186–192. https://doi.org/10.1097/QAI.0000000000000185.
South Group. Assessment report on internet for awareness raising activities. Ghent: CERCA Project; 2014.
Swiss Agency for Development and Cooperation. Planning and monitoring in results-based Management of Projects and Programmes. Zurich: Nadel & ETH; 2011.
Tsu, V., Cernuschi, T. & LaMontagne, D., 2014. Lessons learned from HPV vaccine delivery in low-resource settings and opportunities for HIV prevention, treatment, and care among adolescents. J Acquir Immune Defic Syndr. 2014 Jul 1;66 Suppl 2. pp.209-216. https://doi.org/10.1097/QAI.0000000000000175.
UN Women. How to manage gender-responsive evaluation: evaluation handbook. New York: UN-Women; 2015.
UNDP. Handbook on planning, monitoring and evaluating for development results. New York: UNDP; 2009.
UNFPA. Multilateral organisation performance assessment network (MOPAN-2014) - technical report -volume I: results by micro-indicator and by country. New York: UNFPA; 2014.
UNITAR. Monitoring and evaluation policy framework. Geneva: United Nations Institute for Training and Research; 2012.
United Nations Development Group. Results-based management handbook - harmonizing RBM concepts and approaches for improved development results at country level. Nueva York: United Nations Development Group; 2011.
United Nations, 2014. The Millennium Development Goals Report, New York: United Nations, DPI/2590 E.
UN-Women, 2015. Programming Essentials, Monitoring & Evaluation. 2015. [Accessed 3 Aug 2015]. Available at: http://www.endvawnow.org/en/articles/341-health.html.