Does balancing site characteristics result in balanced population characteristics in a cluster-randomized controlled trial?
Tóm tắt
Intervention trials with nested designs seek to balance sites randomized regarding key site characteristics. Among the goals of such site-level balancing is to accrue patient-level equivalence among treatment arms. We investigated patient-level equivalence in a cluster randomized controlled trial, which balanced study waves on site-level characteristics. The Behavioral Health Interdisciplinary Program—Collaborative Chronic Care Model project utilized a stepped wedge design to stagger implementation of an evidence-based team-oriented mental health patient management system at 9 Veteran Affairs Medical Centers. Study sites were balanced on eight site-level characteristics over time (3 balanced waves [consecutive time periods] with 3 sites per wave) to minimize trend. Sites were balanced on selected site-level characteristics but not on patient-level variables. We explored internal differences in patient demographics across the three study waves. Eligible patients had at least two visits to a participating mental health clinic in the prior year and did not have a diagnosis of dementia (n = 5,596). We found modest but statistically significant inter-site differences in age, marital status, ethnicity, service-related disability, mental health hospitalizations, and selected diagnoses by study wave. Although many of the differences in patient demographics by study wave were statistically significant, only a few results were practically meaningful as measured by effect size. A bipolar diagnosis (49.0%, 21.0%, 17.0% in waves 1–3, respectively; Cramer’s V = 0.3124) and Hispanic ethnicity (2.9%, 29.6%, 2.0% in waves 1–3, respectively; Cramer’s V = 0.3949) resulted in differences that were considered a ‘moderate’ effect size. The number of patient characteristics that were both statistically and meaningfully different by study wave among all possible site assignments was comparable to the 34 most balanced site assignments identified in our balancing algorithm. Using a balancing algorithm to reduce imbalance among site characteristics across time periods did not appear to negatively affect the balance of patient characteristics across sites over time. A site-level balancing algorithm that includes characteristics with a direct relationship to relevant patient-level factors may improve the overall balance across key elements of the study, and aide in the interpretation of results.
Tài liệu tham khảo
Donner, A., Klar, N.: Design and Analysis of Cluster Randomization Trials in Health Research, 1st edn. Oxford University Press Inc., New York (2000)
Brown, C.A., Lilford, R.J.: The stepped wedge trial design: a systematic review. BMC Med Res Methodol 6, 54 (2006)
Silber, J.H., Rosenbaum, P.R., Trudeau, M.E., Even-Shoshan, O., Chen, W., Zhang, X., et al.: Multivariate matching and bias reduction in the surgical outcomes study. Med Care 39(10), 1048–1064 (2001)
Blackford, J.U.: Propensity scores: method for matching on multiple variables in down syndrome research. Intellect Dev Disabil 47(5), 348–357 (2009)
Zhan, C., Miller, M.R.: Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization. JAMA 290(14), 1868–1874 (2003)
Karkouti, K., Beattie, W.S., Dattilo, K.M., McCluskey, S.A., Ghannam, M., Hamdy, A., et al.: A propensity score case-control comparison of aprotinin and tranexamic acid in high-transfusion-risk cardiac surgery. Transfusion 46(3), 327–338 (2006)
Lew, R.A., Miller, C.J., Kim, B., Wu, H., Stolzmann, K., Bauer, M.S.: A method to reduce imbalance for site-level randomized stepped wedge implementation trial designs. Implement Sci 14(1), 46 (2019)
Rosenbaum, P.R., Rubin, D.B.: The bias due to incomplete matching. Biometrics 41(1), 103–116 (1985)
Ming, K., Rosenbaum, P.R.: Substantial gains in bias reduction from matching with a variable number of controls. Biometrics 56(1), 118–124 (2000)
Bauer, M.S., Miller, C., Kim, B., Lew, R., Weaver, K., Coldwell, C., et al.: Partnering with health system operations leadership to develop a controlled implementation trial. Implement Sci 11, 22 (2016)
Bauer, M.S., Miller, C.J., Kim, B., Lew, R., Stolzmann, K., Sullivan, J., et al.: Effectiveness of Implementing a Collaborative Chronic Care Model for Clinician Teams on Patient Outcomes and Health Status in Mental Health: A Randomized Clinical Trial. JAMA Netw Open 2(3), e190230 (2019)
Miller, C.J., Grogan-Kaylor, A., Perron, B.E., Kilbourne, A.M., Woltmann, E., Bauer, M.S.: Collaborative chronic care models for mental health conditions: cumulative meta-analysis and metaregression to guide future research and implementation. Med Care 51(10), 922–930 (2013)
Woltmann, E., Grogan-Kaylor, A., Perron, B., Georges, H., Kilbourne, A.M., Bauer, M.S.: Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am. J. Psychiatry 169(8), 790–804 (2012)
Silber, J.H., Rosenbaum, P.R., Ross, R.N., Ludwig, J.M., Wang, W., Niknam, B.A., et al.: A hospital-specific template for benchmarking its cost and quality. Health Serv Res 49(5), 1475–1497 (2014)
Zubizarreta, J.R., Reinke, C.E., Kelz, R.R., Silber, J.H., Rosenbaum, P.R.: Matching for Several Sparse Nominal Variables in a Case-Control Study of Readmission Following Surgery. Am Stat 65(4), 229–238 (2011)
US Department of Veterans Affairs.: 172VA10P2: VHA Corporate Data Warehouse – VA. 79 FR 4377; Accessed August 15, 2018
Effect Size: Statistics Solutions: Advancement Through Clarity; 2013 [Available from: http://www.statisticssolutions.com/academic-solutions/resources/directory-of-statistical-analyses/effect-size/
Levine, T.R.H.: C.R. Eta squared, partial eta squared, and misreporting of effect size in communication research. Hum Commun Res 28, 612–625 (2002)
Cohen, J.: Statistical power and analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, Inc., Hillsdale (1988)
Wennberg, J.E.: Dealing with medical practice variations: a proposal for action. Health Aff. (Millwood) 3(2), 6–32 (1984)
Luft, H.S.: Becoming accountable-opportunities and obstacles for ACOs. N. Engl. J. Med. 363(15), 1389–1391 (2010)
Kilbourne, A.M., Neumann, M.S., Pincus, H.A., Bauer, M.S., Stall, R.: Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci 2, 42 (2007)
Shortell, S.M., Wu, F.M., Lewis, V.A., Colla, C.H., Fisher, E.S.: A taxonomy of accountable care organizations for policy and practice. Health Serv Res 49(6), 1883–1899 (2014)
Fihn, S.D., Francis, J., Clancy, C., Nielson, C., Nelson, K., Rumsfeld, J., et al.: Insights from advanced analytics at the Veterans Health Administration. Health Aff. (Millwood) 33(7), 1203–1211 (2014)