The “RCT augmentation”: a novel simulation method to add patient heterogeneity into phase III trials
Tóm tắt
Phase III randomized controlled trials (RCT) typically exclude certain patient subgroups, thereby potentially jeopardizing estimation of a drug’s effects when prescribed to wider populations and under routine care (“effectiveness”). Conversely, enrolling heterogeneous populations in RCTs can increase endpoint variability and compromise detection of a drug’s effect. We developed the “RCT augmentation” method to quantitatively support RCT design in the identification of exclusion criteria to relax to address both of these considerations. In the present manuscript, we describe the method and a case study in schizophrenia. We applied typical RCT exclusion criteria in a real-world dataset (cohort) of schizophrenia patients to define the “RCT population” subgroup, and assessed the impact of re-including each of the following patient subgroups: (1) illness duration 1–3 years; (2) suicide attempt; (3) alcohol abuse; (4) substance abuse; and (5) private practice management. Predictive models were built using data from different “augmented RCT populations” (i.e., subgroups where patients with one or two of such characteristics were re-included) to estimate the absolute effectiveness of the two most prevalent antipsychotics against real-world results from the entire cohort. Concurrently, the impact on RCT results of relaxing exclusion criteria was evaluated by calculating the comparative efficacy of those two antipsychotics in virtual RCTs drawing on different “augmented RCT populations”. Data from the “RCT population”, which was defined with typical exclusion criteria, allowed for a prediction of effectiveness with a bias < 2% and mean squared error (MSE) = 5.8–6.8%. Compared to this typical RCT, RCTs using augmented populations provided improved effectiveness predictions (bias < 2%, MSE = 5.3–6.7%), while returning more variable comparative effects. The impact of augmentation depended on the exclusion criterion relaxed. Furthermore, half of the benefit of relaxing each criterion was gained from re-including the first 10–20% of patients with the corresponding real-world characteristic. Simulating the inclusion of real-world subpopulations into an RCT before running it allows for quantification of the impact of each re-inclusion upon effect detection (statistical power) and generalizability of trial results, thereby explicating this trade-off and enabling a controlled increase in population heterogeneity in the RCT design.
Tài liệu tham khảo
Nordon C, Karcher H, Groenwold RH, Ankarfeldt MZ, Pichler F, Chevrou-Severac H, et al. GetReal consortium. GetReal consortium. The “efficacy-effectiveness gap”: historical background and current conceptualization. Value Health. 2016; https://doi.org/10.1016/j.jval.2015.09.2938.
Van Spall HG, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA. 2007;297(11):1233–40.
Heng DY, Choueiri TK, Rini BI, Lee J, Yuasa T, Pal SK, Srinivas S, Bjarnason GA, Knox JJ, Mackenzie M, Vaishampayan UN, Tan MH, Rha SY, Donskov F, Agarwal N, Kollmannsberger C, North S, Wood LA. Outcomes of patients with metastatic renal cell carcinoma that do not meet eligibility criteria for clinical trials. Ann Oncol. 2014; https://doi.org/10.1093/annonc/mdt492.
Rosenheck R, Perlick D, Bingham S, Liu-Mares W, Collins J, Warren S, et al. Department of Veterans Affairs Cooperative Study Group on the cost-effectiveness of olanzapine. Effectiveness and cost of olanzapine and haloperidol in the treatment of schizophrenia. JAMA. 2003;290:2693–702.
Hofer A, Hummer M, Huber R, Kurz M, Walch T, Fleischhacker WW. Selection bias in clinical trials with antipsychotics. J Clin Psychopharmacol. 2000;20:699–702.
Bowen J, Hirsch S. Recruitment rates and factors affecting recruitments for a clinical trial of a putative anti-psychotic agent in the treatment of acute schizophrenia. Hum Psychopharmacol. 1992; https://doi.org/10.1002/hup.470070507.
Robinson D, Woerner MG, Pollack S, Lerner G. Subject selection biases in clinical trials: data from a multicenter schizophrenia treatment study. J Clin Psychopharmacol. 1996;16:170–6.
Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Threats to applicability of randomised trials: exclusions and selective participation. J Health Serv Res Policy. 1999;4(2):112–21.
Horwitz RI, Viscoli CM, Clemens JD, Sadock RT. Developing improved observational methods for evaluating therapeutic effectiveness. Am J Med. 1990;89(5):630–8.
Flather M, Delahunty N, Collinson J. Generalizing results of randomized trials to clinical practice: reliability and cautions. Clin Trials. 2006;3(6):508–12. Review
Panayidou K, Gsteiger S, Egger M, Kilcher G, Carreras M, Efthimiou O, et al. On behalf of the GetReal methods review group. GetReal in mathematical modelling: a review of studies predicting drug effectiveness in the real world. Res Synth Methods. 2016;7(3):264–77.
Pressler TR, Kaizar EE. The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias. Stat Med. 2013;32(20):3552–68. https://doi.org/10.1002/sim.5802. Epub 2013 Apr 1
California: UCSF Clinical & Translational Science Institute. Sample size calculators. http://www.sample-size.net/. Accessed 6 Sept 2017.
Estimating Sample Size and Power. In: Hulley SB, Cummings SR, Browner WS, Grady D, Newman TB, Designing clinical research: an epidemiologic approach. 4th. Philadelphia: Lippincott Williams & Wilkins; 2013. 73.
Comparing Means. In: Chow S-C, Shao J, Wang H. Sample size calculations in clinical research. 2. Boca Raton: Chapman & Hall/CRC; 2008. 58.
Freemantle N, Strack T. Real-world effectiveness of new medicines should be evaluated by appropriately designed clinical trials. J Clin Epidemiol. 2010; https://doi.org/10.1016/j.jclinepi.2009.07.013.
Viscoli C, Bruzzi P, Glauser M. An approach to the design and implementation of clinical trials of empirical antibiotic therapy in febrile and neutropenic cancer patients. Eur J Cancer. 1995;31A:2013–22.
Haro JM, Edgell ET, Jones PB, Alonso J, Gavart S, Gregor KJ, ; SOHO Study Group.et al. The European schizophrenia outpatient health outcomes (SOHO) study: rationale, methods and recruitment. Acta Psychiatr Scand 2003;107:222–232.
Haro JM, Edgell ET, Novick D, Alonso J, Kennedy L, Jones PB, et al. SOHO advisory board. Effectiveness of antipsychotic treatment for schizophrenia: 6-month results of the pan-European schizophrenia outpatient health outcomes (SOHO) study. Acta Psychiatr Scand. 2005;111:220–31.
DDD definition and general considerations. WHO Collaborating Centre for Drug Statistics Methodology 2009 December 17. Available from: http://www.whocc.no/ddd/definition_and_general_considera/. Accessed 6 Sept 2017.
Thornly B, Adams C. Content and quality of 2000 controlled trials in schizophrenia over 50 years. BMJ. 1998;317(7167):1181–4.
Leucht S, Cipriani A, Spineli L, Mavridis D, Orey D, Richter F, et al. Comparative efficacy and tolerability of 15 antipsychotic drugs in schizophrenia: a multiple-treatments meta-analysis. Lancet. 2013; https://doi.org/10.1016/S0140-6736(13)60733-3.
Nordon C, Bovagnet T, Belger M, Jimenez J, Olivares R, Chevrou-Severac H, et al. IMI GetReal WP2 group. Trial exclusion criteria and their impact on the estimation of antipsychotic drugs effect: a case study using the SOHO database. Schizophr Res. 2017; https://doi.org/10.1016/j.schres.2017.07.031.
Bender R, Grouven U. Ordinal logistic regression in medical research. J R Coll Physicians Lond. 1997;31(5):546–51.
R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2013. http://www.R-project.org/
Verde PE, Ohmann C, Morbach S, Icks A. Bayesian evidence synthesis for exploring generalizability of treatment effects: a case study of combining randomized and non-randomized results in diabetes. Stat Med. 2016; https://doi.org/10.1002/sim.6809.
Stroup TS, Alves WM, Hamer RM, Lieberman JA. Clinical trials for antipsychotic drugs: design conventions, dilemmas and innovations. Nat Rev Drug Discov. 2006;5:133–46.
Rudser KD, Bendert E, Koopmeiners JS. Sample size and screening size trade-off in the presence of subgroups with different expected treatment effects. J Biopharm Stat. 2014;24(2):344–58. https://doi.org/10.1080/10543406.2013.860154.
Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011;46(3):399–424. Epub 2011 Jun 8
Meltzer HY, Risinger R, Nasrallah HA, Du Y, Zummo J, Corey L, et al. A randomized, double-blind, placebo-controlled trial of aripiprazole lauroxil in acute exacerbation of schizophrenia. J Clin Psychiatry. 2015; https://doi.org/10.4088/JCP.14m09741.
Kane JM, Peters-Strickland T, Baker RA, Hertel P, Eramo A, Jin N, et al. Aripiprazole once-monthly in the acute treatment of schizophrenia: findings from a 12-week, randomized, double-blind, placebo-controlled study. J Clin Psychiatry. 2014; https://doi.org/10.4088/JCP.14m09168.
Long-Term Efficacy and Safety of Asenapine Using Olanzapine as a Positive Control. 2015. http://clinicaltrials.gov/ (Identification No. NCT00156091). Accessed 11 Aug 2015