Sustaining an Evidence-Based Program Over Time: Moderators of Sustainability and the Role of the Getting to Outcomes® Implementation Support Intervention

Aarons, G. A., Sommerfeld, D. H., & Walrath-Greene, C. M. (2009). Evidence-based practice implementation: The impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implementation Science, 4, 83. https://doi.org/10.1186/1748-5908-4-83.

Acosta, J., Chinman, M., Ebener, P., Malone, P. S., Paddock, S., Phillips, A., et al. (2013). An intervention to improve program implementation: Findings from a two-year cluster randomized trial of assets-getting to outcomes. Implementation Science : IS, 8, 87–87. https://doi.org/10.1186/1748-5908-8-87.

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300.

Campbell, M. K., Carr, C., DeVellis, B., Switzer, B., Biddle, A., Amamoo, M. A., et al. (2009). A randomized trial of tailoring and motivational interviewing to promote fruit and vegetable consumption for cancer prevention and control. [journal article]. Annals of Behavioral Medicine, 38, 71. https://doi.org/10.1007/s12160-009-9140-5.

Cannon, J. S., Gilbert, M., Ebener, P., Malone, P. S., Reardon, C. M., Acosta, J., & Chinman, M. (2019). Influence of an implementation support intervention on barriers and facilitators to delivery of a substance use prevention program. Prevention Science. https://doi.org/10.1007/s11121-019-01037-x.

Chinman, M., Hunter, S. B., Ebener, P., Paddock, S. M., Stillman, L., Imm, P., et al. (2008a). The getting to outcomes demonstration and evaluation: An illustration of the prevention support system. [journal article]. American Journal of Community Psychology, 41, 206–224. https://doi.org/10.1007/s10464-008-9163-2.

Chinman, M., Hunter, S. B., Ebener, P., Paddock, S. M., Stillman, L., Imm, P., et al. (2008b). The getting to outcomes demonstration and evaluation: An illustration of the prevention support system. American Journal of Community Psychology, 41, 206–224. https://doi.org/10.1007/s10464-008-9163-2.

Chinman, M., Tremain, B., Imm, P., & Wandersman, A. (2009). Strengthening prevention performance using technology: A formative evaluation of interactive getting to outcomes. The American Journal of Orthopsychiatry, 79, 469–481. https://doi.org/10.1037/a0016705.

Chinman, M., Acosta, J., Ebener, P., Malone, P. S., & Slaughter, M. E. (2016). Can implementation support help community-based settings better deliver evidence-based sexual health promotion programs? A randomized trial of getting to outcomes®. [journal article]. Implementation Science, 11, 78. https://doi.org/10.1186/s13012-016-0446-y.

Chinman, M., Acosta, J., Ebener, P., Malone, P. S., & Slaughter, M. E. (2018a). A cluster-randomized trial of getting to outcomes' impact on sexual health outcomes in community-based settings. Prevention Science, 19, 437–448. https://doi.org/10.1007/s11121-017-0845-6.

Chinman, M., Hunter, S. B., Ebener, P., Paddock, S. M., Stillman, L., Imm, P., et al. (2008b). The getting to outcomes demonstration and evaluation: An illustration of the prevention support system. American Journal of Community Psychology, 41, 206–224. https://doi.org/10.1007/s10464-008-9163-2.

Chinman, M., Acosta, J., Ebener, P., Hunter, S., Imm, P., & Wandersman, A. (2019). Dissemination of evidence-based prevention interventions and policies. In Z. Sloboda, H. Petras, E. Robertson, & R. Hingson (Eds.), Prevention of substance use (pp. 367–383). Cham: Springer International Publishing.

Cicchetti, D. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instrument in psychology (Vol. 6).

Cooper, B. R., Bumbarger, B. K., & Moore, J. E. (2015). Sustaining evidence-based prevention programs: correlates in a large-scale dissemination initiative. Prevention Science, 16, 145–57.

D'Amico, E. J., Tucker, J. S., Miles, J. N. V., Zhou, A. J., Shih, R. A., & Green Jr., H. D. (2012). Preventing alcohol use with a voluntary after-school program for middle school students: Results from a cluster randomized controlled trial of CHOICE. Prevention Science, 13, 415–425. https://doi.org/10.1007/s11121-011-0269-7.

Damschroder, L. J., & Lowery, J. C. (2013). Evaluation of a large-scale weight management program using the Consolidated Framework for Implementation Research (CFIR). Implementation Science, 8, 51. https://doi.org/10.1186/1748-5908-8-51.

Drug Enforcement Administration. (2017). National drug threat assessment. Washington (DC): Department of Justice (US).

Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105, 399–412. https://doi.org/10.1111/bjop.12046.

Elwert, F., & Winship, C. (2014). Endogenous selection bias: The problem of conditioning on a collider variable. Annual Review of Sociology, 40, 31–53.

Gloppen, K. M., Arthur, M. W., Hawkins, J. D., & Shapiro, V. B. (2012). Sustainability of the communities that care prevention system by coalitions participating in the community youth development study. The Journal of Adolescent Health : Official Publication of the Society for Adolescent Medicine, 51, 259–264. https://doi.org/10.1016/j.jadohealth.2011.12.018.

Hailemariam, M., Bustos, T., Montgomery, B., Barajas, R., Evans, L. B., & Drahota, A. (2019). Evidence-based intervention sustainability strategies: A systematic review. Implementation Science, 14, 57–68.

Johnson, K., Collins, D., Shamblen, S., Kenworthy, T., & Wandersman, A. (2017). Long-term sustainability of evidence-based prevention interventions and community coalitions survival: A five and one-half year follow-up study. Prevention Science, 18, 610–621. https://doi.org/10.1007/s11121-017-0784-2.

Johnston, L. D., Miech, R. A., O'Malley, P. M., Bachman, J. G., Schulenberg, J. E., & Patrick, M. E. (2019). Monitoring the future national survey results on drug use, 1975–2018: Overview, Key Findings on Adolescent Drug Use. Institute for Social Research.

Kirchner, J. E., Ritchie, M. J., Pitcock, J. A., Parker, L. E., Curran, G. M., & Fortney, J. C. (2014). Outcomes of a partnered facilitation strategy to implement primary care-mental health. Journal of General Internal Medicine, 29, 904–912. https://doi.org/10.1007/s11606-014-3027-2.

Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Sage. Organizational Research Methods, 13, 392–394.

Moore, J. E., Mascarenhas, A., Bain, J., & Straus, S. E. (2017). Developing a comprehensive definition of sustainability. Implementation Science, 12, 110–117.

Moyers, T., Martin, T., Manuel, J., Miller, W., & Ernst, D. (2010). Revised global scales: Motivational interviewing treatment integrity 3.1. 1 (MITI 3.1. 1). Unpublished manuscript, University of New Mexico, Albuquerque, NM.

Peterson, A. E., Bond, G. R., Drake, R. E., McHugo, G. J., Jones, A. M., & Williams, J. (2014). Predicting the long-term sustainability of evidence-based practices in mental health care: An 8-year longitudinal analysis. The Journal of Behavioral Health Services & Research, 41, 337–346. https://doi.org/10.1007/s11414-013-9347-x.

Pollak, K. I., Coffman, C. J., Alexander, S. C., Østbye, T., Lyna, P., Tulsky, J. A., et al. (2014). Weight's up? Predictors of weight-related communication during primary care visits with overweight adolescents. Patient Education and Counseling, 96, 327–332. https://doi.org/10.1016/j.pec.2014.07.025.

Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of implementation strategies: Results from the expert recommendations for implementing change (ERIC) project. Implementation Science, 10, 21–34.

Rycroft-Malone, J., Harvey, G., Seers, K., Kitson, A., McCormack, B., & Titchen, A. (2004). An exploration of the factors that influence the implementation of evidence into practice. Journal of Clinical Nursing, 13, 913–924. https://doi.org/10.1111/j.1365-2702.2004.01007.x.

Sacks, J. J., Gonzales, K. R., Bouchery, E. E., Tomedi, L. E., & Brewer, R. D. (2015). 2010 national and state costs of excessive alcohol consumption. American Journal of Preventive Medicine, 49, e73–e79.

Shelton, R. C., & Lee, M. (2019). Sustaining evidence-based interventions and policies: Recent innovations and future directions in implementation science. Annual Review of Public Health, 39, 55–76.

Shelton, R. C., Cooper, B. R., & Stirman, S. W. (2018). The sustainability of evidence-based interventions and practices in public health and health care. Annual Review of Public Health, 39, 55–76.

Spoth, R., Guyll, M., Redmond, C., Greenberg, M., & Feinberg, M. (2011). Six-year sustainability of evidence-based intervention implementation quality by community-university partnerships: The PROSPER study. [journal article]. American Journal of Community Psychology, 48, 412–425. https://doi.org/10.1007/s10464-011-9430-5.

Stirman, S., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7, 17. https://doi.org/10.1186/1748-5908-7-17.

Textor, J., van der Zander, B., Gilthorpe, M. K., Liskiewicz, M., & Ellison, G. T. H. (2015). Welcome to DAGitty [on-line]. Accessed November 6, 2018, from http://dagitty.net/

Textor, J., van der Zander, B., Gilthorpe, M. K., Liskiewicz, M., & Ellison, G. T. H. (2016). Robust causal inference using directed acyclic graphs: The R package ‘dagitty’. International Journal of Epidemiology, 45, 1887–1894.

Turrisi, R., Larimer, M. E., Mallett, K. A., Kilmer, J. R., Ray, A. E., Mastroleo, N. R., et al. (2009). A randomized clinical trial evaluating a combined alcohol intervention for high-risk college students. Journal of Studies on Alcohol and Drugs, 70, 555–567.

Wandersman, A., & Florin, P. (2003). Community interventions and effective prevention. The American Psychologist, 58, 441–448.

Welsh, J. A., Chilenski, S. M., Johnson, L., Greenberg, M. T., & Spoth, R. L. (2016). Pathways to sustainability: 8-year follow-up from the PROSPER project. The Journal of Primary Prevention, 37, 263–286.

Wiseman, S. H., Chinman, M., Ebener, P. A., Hunter, S., Imm, P. S., & Wandersman, A. H. (2007). Getting to outcomes: 10 steps for achieving results-based accountability. Santa Monica, CA: RAND.