Validation and refinement of the Stakeholder-driven Community Diffusion Survey for childhood obesity prevention
Tóm tắt
Whole-of-community interventions hold promise in addressing childhood obesity. The COMPACT Stakeholder-driven Community Diffusion theory posits that stakeholders’ knowledge of childhood obesity prevention efforts and engagement with the issue contribute to successful intervention implementation. Building on completed formative research and pilot testing, we describe the validation and refinement of knowledge and engagement measures. We assessed content validity using a modified Delphi process with science (n=18) and practice-based (n=16) experts. Next, we refined the survey based on input from science- and practice-based experts, cognitive response testing, and item analysis of extant survey data. Field testing of the refined survey involved community stakeholders in Greenville County, South Carolina (n=50), East Boston, Massachusetts (n=30), and Tucson, Arizona (n=84) between 2019 and 2020. Construct validity was assessed with confirmatory factor analysis (CFA). Two-week test-retest reliability was assessed among a subsample of 14 paired respondents in South Carolina. Experts rated existing knowledge domains (intervention factors, roles, sustainability, problem, resources) and engagement domains (dialogue/mutual learning, flexibility, influence/power, leadership/stewardship, trust) highly for their importance in addressing childhood obesity. Expert input resulted in 11 new knowledge items and 7 new engagement items that mapped onto existing domains. Correspondingly, two domain names were modified: implementation/sustainability and trust/trustworthiness. We also eliminated 8 extant items (4 knowledge and 4 engagement) and adapted item language for comprehension and consistency. Further modifications based on CFA results and item analyses resulted in 23 knowledge items across four domains (roles and resources merged) and 23 engagement items across five domains. Modified knowledge and engagement scales had adequate fit and strong item factor loadings (most >0.7 and all >0.5). Knowledge (α=0.86–0.87) and engagement (α=0.75–0.90) subscales had high internal scale consistency. Knowledge intraclass correlation coefficients (ICCs) for test-retest agreement of subscale scores ranged from 0.50 for intervention factors to 0.86 for roles/resources. For engagement subscale scores, ICCs ranged from 0.70 for trust/trustworthiness to 0.96 for leadership/stewardship. Findings from this multi-method survey development process increase our confidence of the knowledge and engagement measures’ content validity, construct validity, and reliability.
Tài liệu tham khảo
IOM (Institute of Medicine). Accelerating Progress in Obesity Prevention: Solving the Weight of the Nation. Washington, DC: The National Academies Press; 2012.
The National Academies of Sciences Engineering Medicine, Health and Medicine Division. Driving Action and Progress on Obesity Prevention and Treatment: Proceedings of a Workshop. Washington, DC: The National Academies Press; 2017.
National Academies of Sciences, Engineering, Medicine. In: Olson S, editor. Obesity in the Early Childhood Years: State of the Science and Implementation of Promising Solutions: Workshop Summary. Washington, DC: The National Academies Press; 2016. p. 114.
Mazzucca S, Arredondo EM, Hoelscher DM, Haire-Joshu D, Tabak RG, Kumanyika SK, et al. Expanding Implementation research to prevent chronic diseases in community settings. Annual Review of Public Health. 2021;42:135.
Korn AR, Hennessy E, Hammond RA, Allender S, Gillman MW, Kasman M, et al. Development and testing of a novel survey to assess Stakeholder-driven Community Diffusion of childhood obesity prevention efforts. BMC Public Health. 2018;18(1):681. https://doi.org/10.1186/s12889-018-5588-1.
Appel JM, Fullerton K, Hennessy E, Korn AR, Tovar A, Allender S, et al. Design and methods of Shape Up Under 5: Integration of systems science and community-engaged research to prevent early childhood obesity. PLOS ONE. 2019;14(8):e0220169. https://doi.org/10.1371/journal.pone.0220169.
Kasman M, Hammond RA, Heuberger B, Mack-Crane A, Purcell R, Economos C, et al. Activating a community: an agent-based model of Romp & Chomp, a whole-of-community childhood obesity intervention. Obesity (Silver Spring). 2019;27(9):1494–502. https://doi.org/10.1002/oby.22553.
Calancie L, Fullerton K, Appel JM, Korn AR, Hennessy E, Hovmand P, et al. Implementing Group Model Building With the Shape Up Under 5 Community Committee Working to Prevent Early Childhood Obesity in Somerville, Massachusetts. J Public Health Manag Pract. 2020. https://doi.org/10.1097/PHH.0000000000001213.
Korn AR, Hammond RA, Hennessy E, Must A, Pachucki MC, Economos CD. Evolution of a coalition network during a whole-of-community intervention to prevent early childhood obesity. Child Obes. 2021. https://doi.org/10.1089/chi.2020.0156.
Economos CD, Hyatt RR, Goldberg JP, Must A, Naumova EN, Collins JJ, et al. A community intervention reduces BMI z-score in children: Shape Up Somerville first year results. Obesity (Silver Spring, Md). 2007;15(5):1325-36.
de Silva-Sanigorski AM, Bell AC, Kremer P, Nichols M, Crellin M, Smith M, et al. Reducing obesity in early childhood: results from Romp & Chomp, an Australian community-wide intervention program. The American Journal of Clinical Nutrition. 2010;91(4):831–40. https://doi.org/10.3945/ajcn.2009.28826.
adams j. Gathering Social Network Data. Thousand Oaks: SAGE Publications; 2019.
Linstone HA, Turoff M. The Delphi Method: Techniques and Applications. Addison-Wesley Publishing Company, Advanced Book Program; 1975.
Willis G. Cognitive Interviewing. Thousand Oaks: California SAGE Publications, Inc.; 2005. Available from: http://methods.sagepub.com/book/cognitive-interviewing. https://doi.org/10.4135/9781412983655.
Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best Practices for developing and validating scales for health, social, and behavioral research: a primer. Frontiers in Public Health. 2018;6:149.
Sapsford R. Constructing scales. Survey Research. 2nd ed. London: SAGE Publications, Ltd; 2007. p. 222–45.
Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Journal of Clinical Epidemiology. 2008;61(2):102–9. https://doi.org/10.1016/j.jclinepi.2007.03.012.
Hays RD, Hadorn D. Responsiveness to change: an aspect of validity, not a separate dimension. Quality of Life Research. 1992;1(1):73–5. https://doi.org/10.1007/BF00435438.
Fok CCT, Henry D. Increasing the sensitivity of measures to change. Prevention Science: The Official Journal of the Society for Prevention Research. 2015;16(7):978–86. https://doi.org/10.1007/s11121-015-0545-z.
ChildObesity180 at Tufts University. Catalyzing Communities: A ChildObesity180 Initiative. Available from: https://catalyzingcommunities.org/. Accessed 27 Sept 2021.
Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6(1):1–55. https://doi.org/10.1080/10705519909540118.
Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63. https://doi.org/10.1016/j.jcm.2016.02.012.
Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6(4):284–90. https://doi.org/10.1037/1040-3590.6.4.284.
Plested BA, Edwards RW, Jumper-Thurman P. Community readiness: a handbook for successful change. Tri-Ethnic Center for Prevention Research: Fort Collins, CO; 2006.
Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. New York: Oxford University Press; 2015.
Boston College School of Social Work: Center for Social Innovation. Systems Science. Available from: https://www.bc.edu/content/bc-web/schools/ssw/sites/center-for-social-innovation/systems-science.html. Accessed Sept 27 2021.