Treatment decisions in multiple sclerosis — insights from real-world observational studies

Nature Reviews Neurology - Tập 13 Số 2 - Trang 105-118 - 2017
Maria Trojano1, Mar Tintoré2, Xavier Montalbán2, Jan Hillert3, Tomáš Kalinčík4, Pietro Iaffaldano1, Tim Spelman4, Maria Pia Sormani5, Helmut Butzkueven4
1Department of Basic Medical Sciences, Neurosciences and Sense Organs, University of Bari "Aldo Moro", Piazza G. Cesare 11, Bari, 70124, Italy
2Department of Neurology-Neuroimmunology and Multiple Sclerosis Centre of Catalonia (Cemcat), Hospital Universitari Vall d'Hebron, Universitat Autònoma de Barcelona, Passeig Vall d'Hebron 119–129, Barcelona, 08035, Spain
3Department of Clinical Neuroscience, Karolinska Institutet, Tomtebodavägen 18A, Stockholm, S-17177, Solna, Sweden
4Department of Medicine, and Department of Neurology, University of Melbourne, Royal Melbourne Hospital, Grattan Street, Parkville, 3050, VIC, Australia
5Department of Health Sciences (DISSAL), University of Genoa, Via Pastore 1, Genoa 16132, Italy.

Tóm tắt

Từ khóa


Tài liệu tham khảo

Rothwell, P. M. External validity of randomised controlled trials: “to whom do the results of this trial apply?” Lancet 365, 82–93 (2005).

Dekkers, O. M., von Elm, E., Algra, A., Romijn, J. A. & Vandenbroucke, J. P. How to assess the external validity of therapeutic trials: a conceptual approach. Int. J. Epidemiol. 39, 89–94 (2010).

Sormani, M. P. & Bruzzi, P. Can we measure long-term treatment effects in multiple sclerosis? Nat. Rev. Neurol. 11, 176–182 (2015).

Kalincik, T. & Butzkueven, H. Observational data: understanding the real MS world. Mult. Scler. 22, 1642–1648 (2016).

Ioannidis, J. P. Indirect comparisons: the mesh and mess of clinical trials. Lancet 368, 1470–1472 (2006).

Van Luijn, J. C., Gribnau, F. W. & Leufkens, H. G. Availability of comparative trials for the assessment of new medicines in the European Union at the moment of market authorization. Br. J. Clin. Pharmacol. 63, 159–162 (2007).

Confavreux, C., Compston, D. A., Hommes, O. R., McDonald, W. I. & Thompson, A. J. EDMUS, a European database for multiple sclerosis. J. Neurol. Neurosurg. Psychiatry 55, 671–676 (1992).

Butzkueven, H. et al. MSBase: an international, online registry and platform for collaborative outcomes research in multiple sclerosis. Mult. Scler. 12, 769–774 (2006).

Trojano, M. et al. Italian Multiple Sclerosis Database Network. Neurol. Sci. 27 (Suppl. 5), S358–S361 (2006).

Myhr, K. M., Grytten, N. & Aarseth, J. H. The Norwegian Multiple Sclerosis Registry and Biobank. Acta Neurol. Scand. Suppl. 195, 20–23 (2012).

Flachenecker, P. et al. EUReMS Consortium. Multiple sclerosis registries in Europe — results of a systematic survey. Mult. Scler. 20, 1523–1532 (2014).

Hillert, J. & Stawiarz, L. The Swedish MS registry — clinical support tool and scientific resource. Acta Neurol. Scand. 132, 11–19 (2015).

Koch-Henriksen, N., Magyari, M. & Laursen, B. Registers of multiple sclerosis in Denmark. Acta Neurol. Scand. 132, 4–10 (2015).

Cook, J. A. & Collins, G. S. The rise of big clinical databases. Br. J. Surg. 102, e93–e101 (2015).

Benson, K. & Hartz, A. J. A comparison of observational studies and randomized, controlled trials. N. Engl. J. Med. 342, 1878–1886 (2000).

Concato, J., Shah, N. & Horwitz, R. I. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N. Engl. J. Med. 342, 1887–1892 (2000).

Porter, M. E., Larsson, S. & Lee, T. H. Standardizing patient outcomes measurement. N. Engl. J. Med. 374, 504–506 (2016).

Kitsios, G. D. Propensity score studies are unlikely to underestimate treatment effects in critical care medicine: a critical reanalysis. J. Clin. Epidemiol. 68, 467–469 (2015).

Rosenbaum, P. R. & Rubin, D. B. The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983).

Austin, P. C. The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies. Med. Decis. Making 29, 661–677 (2009).

Trojano, M. et al. Observational studies: propensity score analysis of non-randomized data. Int. MS J. 16, 90–97 (2009).

Trojano, M. in Multiple Sclerosis Therapeutics 4th edn Ch. 21 (eds Cohen, J. A. & Rudick, R.) 244–252 (Cambridge Univ. Press, 2011).

Glynn, R. J., Schneeweiss, S. & Stürmer, T. Indications for propensity scores and review of their use in pharmacoepidemiology. Basic Clin. Pharmacol. Toxicol. 98, 253–259 (2006).

Austin, P. C. Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: a systematic review and suggestions for improvement. J. Thorac. Cardiovasc. Surg. 134, 1128–1135 (2007).

Stenestrand, U., Wallentin, L. & Swedish Register of Cardiac Intensive Care (RIKS-HIA). Early statin treatment following acute myocardial infarction and 1-year survival. JAMA 285, 430–436 (2001).

Gum, P. A., Thamilarasan, M., Watanabe, J., Blackstone, E. H. & Lauer, M. S. Aspirin use and all-cause mortality among patients being evaluated for known or suspected coronary artery disease: a propensity analysis. JAMA 286, 1187–1194 (2001).

Kern, L. M. et al. Association between screening for osteoporosis and the incidence of hip fracture. Ann. Intern. Med. 142, 173–181 (2005).

Schneeweiss, S. et al. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 20, 512–522 (2009).

Rosenbaum, P. R. Discussing hidden bias in observational studies. Ann. Intern. Med. 115, 901–905 (1991).

Lin, D. Y., Psaty, B. M. & Krommal, R. A. Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics 54, 948–963 (1998).

Karim, M. E. et al. Marginal structural Cox models for estimating the association between β-interferon exposure and disease progression in a multiple sclerosis cohort. Am. J. Epidemiol. 180, 160–171 (2014).

Westreich, D., Cole, S. R., Schisterman, E. F. & Platt, R. W. A simulation study of finite-sample properties of marginal structural Cox proportional hazards models. Stat. Med. 31, 2098–2109 (2012).

Havercroft, W. G. & Didelez, V. Simulating from marginal structural models with time-dependent confounding. Stat. Med. 31, 4190–4206 (2012).

Debray, T., Moons, K. G., Ahmed, I., Koffijberg, H. & Riley, R. D. A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis. Stat. Med. 32, 3158–3180 (2013).

Ahmed, I., Debray, T. P., Moons, K. G. & Riley, R. D. Developing and validating risk prediction models in an individual participant data meta-analysis. BMC Med. Res. Methodol. 14, 3 (2014).

Zhao, L., Tian, L., Cai, T., Claggett, B. & Wei, L. J. Effectively selecting a target population for a future comparative study. J. Am. Stat. Assoc. 108, 527–539 (2013).

Verde, P. E., Ohmann, C., Morbach, S. & Icks, A. Bayesian evidence synthesis for exploring generalizability of treatment effects: a case study of combining randomized and non-randomized results in diabetes. Stat. Med. 35, 1654–1675 (2016).

Scott, I. A. & Attia, J. Cautionary tales in the interpretation of observational studies of effects of clinical interventions. Intern. Med. J. http://dx.doi.org/10.1111/imj.13167 (2016). This article proposes criteria for identifying high quality observational studies.

Arts, D. G., De Keizer, N. F. & Scheffer, G. J. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J. Am. Med. Inform. Assoc. 9, 600–611 (2002).

Christiansen, D. H., Hosking, J. D., Dannenberg, A. L. & Williams, O. D. Computer-assisted data collection in multicenter epidemiologic research. The Atherosclerosis Risk Communities Study. Control. Clin. Trials 11, 101–115 (1990).

Weiskopf, N. G. & Weng, C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J. Am. Med. Inform. Assoc. 20, 144–151 (2013).

Goudar, S. et al. Data quality monitoring and performance metrics of a prospective, population-based observational study of maternal and newborn health in low resource settings. Reprod. Health 12 (Suppl. 2), S2 (2015).

Kalincik, T. et al. Data quality evaluation for observational multiple sclerosis registries. Mult. Scler. http://dx.doi.org/10.1177/1352458516662728 (2016).

Tintore, M. et al. Defining high, medium and low impact prognostic factors for developing multiple sclerosis. Brain 138, 1863–1874 (2015). The largest single-centre prospective study evaluating prognostic factors in patients with clinically isolated syndrome suggestive of MS.

Jokubaitis, V. G. et al. Predictors of disability worsening in clinically isolated syndrome. Ann. Clin. Transl Neurol. 2, 479–491 (2015).

Marrie, R. A. et al. Recommendations for observational studies of comorbidity in multiple sclerosis. Neurology 86, 1446–1453 (2016).

Trojano, M. et al. Real-life impact of early interferonβ therapy in relapsing multiple sclerosis. Ann. Neurol. 66, 513–520 (2009).

Río, J. et al. Defining the response to interferon-beta in relapsing-remitting multiple sclerosis patients. Ann. Neurol. 59, 344–352 (2006).

Río, J. et al. Measures in the first year of therapy predict the response to interferon beta in MS. Mult. Scler. 15, 848–853 (2009).

Bermel, R. A. et al. Predictors of long-term outcome in multiple sclerosis patients treated with interferon β. Ann. Neurol. 73, 95–103 (2013).

Jokubaitis, V. G. et al. Predictors of long-term disability accrual in relapse-onset multiple sclerosis. Ann. Neurol. 80, 89–100 (2016).

Horakova, D. et al. Early predictors of non-response to interferon in multiple sclerosis. Acta Neurol. Scand. 126, 390–397 (2012).

Uher, T. et al. Combining clinical and MRI predictors enhances prediction of 12-year disability in multiple sclerosis. Mult. Scler. http://dx.doi.org/10.1177/1352458516642314 (2016).

Uher, T. et al. Early magnetic resonance imaging predictors of clinical progression after 48 months in clinically isolated syndrome patients treated with intramuscular interferon β-1a. Eur. J. Neurol. 22, 1113–1123 (2015).

Río, J. et al. Evaluating the response to glatiramer acetate in relapsing-remitting multiple sclerosis (RRMS) patients. Mult. Scler. 20, 1602–1608 (2014).

Sormani, M. P. et al. Scoring treatment response in patients with relapsing multiple sclerosis. Mult. Scler. 19, 605–612 (2013).

Prosperini, L. et al. Interferon beta failure predicted by EMA criteria or isolated MRI activity in multiple sclerosis. Mult. Scler. 20, 566–576 (2014).

Dobson, R., Rudick, R. A., Turner, B., Schmierer, K. & Giovannoni, G. Assessing treatment response to interferon-β: is there a role for MRI? Neurology 82, 248–254 (2014).

Sormani, M. P. et al. Assessing response to interferon-β in a multicenter dataset of patients with MS. Neurology 87, 134–140 (2016). The largest study assessing MRI criteria for predicting IFN-β treatment non-response in real-world studies.

Rio, J. et al. Clinical markers of long-term disability in RRMS patients treated with interferon beta [poster]. Mult. Scler. 20 (Suppl. 1), P285 (2014).

Río, J. et al. Relationship between MRI lesion activity and response to IFN-beta in relapsing-remitting multiple sclerosis patients. Mult. Scler. 14, 479–484 (2008).

Altay, E. E. et al. Reliability of classifying multiple sclerosis disease activity using magnetic resonance imaging in a multiple sclerosis clinic. JAMA Neurol. 70, 338–344 (2013).

Wattjes, M. P. et al. Evidence-based guidelines: MAGNIMS consensus guidelines on the use of MRI in multiple sclerosis — establishing disease prognosis and monitoring patients. Nat. Rev. Neurol. 11, 597–606 (2015).

Rudick, R. A., Lee, J. C., Simon, J., Ransohoff, R. M. & Fisher, E. Defining interferon beta response status in multiple sclerosis patients. Ann. Neurol. 56, 548–555 (2004).

Bevan, C. J. & Cree, B. A. Disease activity free status: a new end point for a new era in multiple sclerosis clinical research? JAMA Neurol. 71, 269–270 (2014).

Kappos, L. et al. Inclusion of brain volume loss in a revised measure of 'no evidence of disease activity' (NEDA-4) in relapsing–remitting multiple sclerosis. Mult. Scler. 22, 1297–1305 (2016).

Sormani, M. P., Arnold, D. L. & De Stefano, N. Treatment effect on brain atrophy correlates with treatment effect on disability in multiple sclerosis. Ann. Neurol. 75, 43–49 (2014).

Stangel, M., Penner, I., Kallmann, B. A., Lukas, C. & Kieseier, B. C. Towards the implementation of 'no evidence of disease activity' in multiple sclerosis treatment: the multiple sclerosis decision model. Ther. Adv. Neurol. Disord. 8, 3–13 (2015).

Kuhle, J. et al. Fingolimod and CSF neurofilament light chain levels in relapsing–remitting multiple sclerosis. Neurology 84, 1639–1643 (2015).

Rotstein, D. L., Healy, B. C., Malik, M. T., Chitnis, T. & Weiner, H. L. Evaluation of no evidence of disease activity in a 7-year longitudinal multiple sclerosis cohort. JAMA Neurol. 72, 152–158 (2015).

Jacobs, L. D. et al. Intramuscular interferon beta-1a therapy initiated during a first demyelinating event in multiple sclerosis. CHAMPS Study Group. N. Engl. J. Med. 343, 898–904 (2000).

Comi, G. et al. Effect of early interferon treatment on conversion to definite multiple sclerosis: a randomised study. Lancet 357, 1576–1582 (2001).

Kappos, L. et al. Long-term subcutaneous interferon beta-1a therapy in patients with relapsing-remitting MS. Neurology 67, 944–953 (2006).

Polman, C. et al. Subgroups of the BENEFIT study: risk of developing MS and treatment effect of interferon beta-1b. J. Neurol. 255, 480–487 (2008).

Comi, G. et al. Effect of glatiramer acetate on conversion to clinically definite multiple sclerosis in patients with clinically isolated syndrome (PreCISe study): a randomised, double-blind, placebo-controlled trial. Lancet 374, 1503–1511 (2009).

Kappos, L. et al. Long-term effect of early treatment with interferon beta-1b after a first clinical event suggestive of multiple sclerosis: 5-year active treatment extension of the phase 3 BENEFIT trial. Lancet Neurol. 8, 987–997 (2009).

Kinkel, P. R. et al. Association between immediate initiation of intramuscular interferon beta-1a at the time of a clinically isolated syndrome and long-term outcomes: a 10-year follow-up of the Controlled High-Risk Avonex Multiple Sclerosis Prevention Study in Neurological Surveillance. Arch. Neurol. 69, 183–190 (2012).

Miller, A. E. et al. Oral teriflunomide for patients with a first clinical episode suggestive of multiple sclerosis (TOPIC): a randomised, double-blind, placebo-controlled, phase 3 trial. Lancet Neurol. 13, 977–986 (2014).

Boster, A. et al. Disease activity in the first year predicts longer-term clinical outcomes in the pooled population of the phase III FREEDOMS and FREEDOMS II studies [poster]. Neurology 84 (14 Suppl.), P7.239 (2015).

Giovannoni, G. et al. Is it time to target no evident disease activity (NEDA) in multiple sclerosis? Mult. Scler. Relat. Disord. 4, 329–333 (2015).

Filippini, G. et al. Immunomodulators and immunosuppressants for multiple sclerosis: a network meta-analysis. Cochrane Database Syst. Rev. 6, CD008933 (2013).

Kalincik, T. et al. Comparative effectiveness of glatiramer acetate and interferon beta formulations in relapsing-remitting multiple sclerosis. Mult. Scler. 21, 1159–1171 (2015).

Sekhon, J. Multivariate and propensity score matching software with automated balance optimization: the matching package for R. J. Stat. Software 42, 7 (2011).

Spelman, T. et al. Comparative efficacy of first-line natalizumab versus IFNβ or glatiramer acetate in relapsing-remitting MS. Neurol. Clin. Pract. 6, 102–115 (2016).

Rassen, J. A. et al. One-to-many propensity score matching in cohort studies. Pharmacoepidemiol. Drug Saf. 21 (Suppl. 2), 69–80 (2012).

Winkelmann, A., Loebermann, M., Reisinger, E. C., Hartung, H. P. & Zettl, U. K. Disease-modifying therapies and infectious risks in multiple sclerosis. Nat. Rev. Neurol. 12, 217–233 (2016).

Lublin, F. D., Baier, M. & Cutter, G. Effect of relapses on development of residual deficit in multiple sclerosis. Neurology 61, 1528–1532 (2003).

Hirst, C. et al. Contribution of relapses to disability in multiple sclerosis. J. Neurol. 255, 280–287 (2008).

Stewart, T. et al. Contribution of different relapse phenotypes to disability in multiple sclerosis. Mult. Scler. http://dx.doi.org/10.1177/1352458516643392 (2016).

Khatri, B. et al. Comparison of fingolimod with interferon beta-1a in relapsing-remitting multiple sclerosis: a randomised extension of the TRANSFORMS study. Lancet Neurol. 10, 520–529 (2011).

Coles, A. J. et al. Alemtuzumab for patients with relapsing multiple sclerosis after disease-modifying therapy: a randomised controlled phase 3 trial. Lancet 380, 1829–1839 (2012).

Spelman, T. et al. Comparative efficacy of switching to natalizumab in active multiple sclerosis. Ann. Clin. Transl Neurol. 2, 373–387 (2015).

He, A. et al. Comparison of switch to fingolimod or interferon beta/glatiramer acetate in active multiple sclerosis. JAMA Neurol. 72, 405–413 (2015).

Schoenfeld, D. Chi-squared goodness-of-fit tests for the proportional hazards regression model. Biometrika 67, 145–153 (1980).

Kalincik, T. et al. Switch to natalizumab versus fingolimod in active relapsing-remitting multiple sclerosis. Ann. Neurol. 77, 425–435 (2015). The first comparative study evaluating the effectiveness of natalizumab and fingolimod after first-line treatment failure.

Baroncini, D. et al. Natalizumab versus fingolimod in patients with relapsing-remitting multiple sclerosis non-responding to first-line injectable therapies. Mult. Scler. 22, 1315–1326 (2016).

Barbin, L. et al. Comparative efficacy of fingolimod versus natalizumab: a French multicenter observational study. Neurology 86, 771–778 (2016).

Koch-Henriksen, N., Magyari, M., Sellebjerg, F. & Soelberg Sorensen, P. A comparison of multiple sclerosis clinical disease activity between patients treated with natalizumab and fingolimod. Mult. Scler. http://dx.doi.org/10.1177/1352458516643393 (2016).

Spelman, T. et al. Risk of early relapse following the switch from injectables to oral agents for multiple sclerosis. Eur. J. Neurol. 23, 729–736 (2016).

Bloomgren, G. et al. Risk of natalizumab-associated progressive multifocal leukoencephalopathy. N. Engl. J. Med. 366, 1870–1880 (2012).

O'Connor, P. W. et al. Disease activity return during natalizumab treatment interruption in patients with multiple sclerosis. Neurology 76, 1858–1865 (2011).

Cohen, M. et al. Switching from natalizumab to fingolimod in multiple sclerosis: a French prospective study. JAMA Neurol. 71, 436–441 (2014).

Iaffaldano, P. et al. Fingolimod versus interferon beta/glatiramer acetate after natalizumab suspension in multiple sclerosis. Brain 138, 3275–3286 (2015). The first comparative study demonstrating the superiority of fingolimod versus BRACE therapy in controlling diseases reactivation after natalizumab suspension in a real-world context.

Sorensen, P. S. et al. Recurrence or rebound of clinical relapses after discontinuation of natalizumab therapy in highly active MS patients. J. Neurol. 261, 1170–1177 (2014).

Clerico, M. et al. Treatment of relapsing–remitting multiple sclerosis after 24 doses of natalizumab: evidence from an Italian spontaneous, prospective, and observational study (the TY-STOP Study). JAMA Neurol. 71, 954–960 (2014).

Jokubaitis, V. G. et al. Fingolimod after natalizumab and the risk of short-term relapse. Neurology 82, 1204–1211 (2014).

Alping, P. et al. Rituximab versus fingolimod after natalizumab in multiple sclerosis patients. Ann. Neurol. 79, 950–958 (2016).

Parsons, L. S. Reducing bias in a propensity score matched pair sample using greedy matching techniques. SAS http://www2.sas.com/proceedings/sugi26/p214-26.pdf (2001).

Trojano, M. et al. New natural history of interferon-beta-treated relapsing multiple sclerosis. Ann. Neurol. 61, 300–306 (2007). The first study addressing the issue of long-term effectiveness of IFN-β treatment in MS by using propensity score technique.

Lunceford, J. K. & Davidian, M. Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat. Med. 23, 2937–2960 (2004).

Bergamaschi, R. et al. Immunomodulatory therapies delay disease progression in multiple sclerosis. Mult. Scler. 22, 1732–1740 (2016).

Gilks, R. & Berzuini, C. Following a moving target — Monte Carlo inference for dynamic Bayesian models. J. R. Stat. Soc. B 63, 127–146 (2001).

Tedeholm, H. et al. Time to secondary progression in patients with multiple sclerosis treated with first generation immunomodulating drugs. Mult. Scler. 19, 765–774 (2013).

Shirani, A. et al. Association between use of interferon beta and progression of disability in patients with relapsing–remitting multiple sclerosis. JAMA 308, 247–256 (2012).

Palace, J. et al. Effectiveness and cost-effectiveness of interferon beta and glatiramer acetate in the UK Multiple Sclerosis Risk Sharing Scheme at 6 years: a clinical cohort study with natural history comparator. Lancet Neurol. 14, 497–505 (2015). The first study assessing cost–utility ratios and cost-effectiveness in patients with MS treated with BRACE therapies over a 6-year period.

Craig, B. A. & Sendi, P. P. Estimation of the transition matrix of a discrete-time Markov chain. Health Econ. 11, 33–42 (2002).

Jackson, C. H., Sharples, L. S., Thompson, S. G. & Couto, E. Multistate Markov models for disease progression with classification error. J. R. Stat. Soc. D 52, 193–209 (2003).

Kalincik, T. et al. Defining reliable disability outcomes in multiple sclerosis. Brain 138, 3287–3298 (2015).

Lorscheider, J. et al. Defining secondary progressive multiple sclerosis. Brain 139, 2395–2405 (2016).

Ziemssen, T., Kern, R. & Cornelissen, C. The PANGAEA study design — a prospective, multicenter, non-interventional, long-term study on fingolimod for the treatment of multiple sclerosis in daily practice. BMC Neurol. 15, 93 (2015).

Linker, R. A. & Wendt, G. Cardiac safety profile of first dose of fingolimod for relapsing–remitting multiple sclerosis in real-world settings: data from a German prospective multi-center observational study. Neurol. Ther. http://dx.doi.org/10.1007/s40120-016-0051-7 (2016).

Miclea, A. et al. Safety and efficacy of dimethyl fumarate in multiple sclerosis: a multi-center observational study. J. Neurol. 263, 1626–1632 (2016).

Frisell, T. et al. Comparative analysis of first-year fingolimod and natalizumab drug discontinuation among Swedish patients with multiple sclerosis. Mult. Scler. 22, 85–93 (2016).

Butzkueven, H. et al. Efficacy and safety of natalizumab in multiple sclerosis: interim observational programme results. J. Neurol. Neurosurg. Psychiatry 85, 1190–1197 (2014).

Zhang, T. et al. Examining the effects of comorbidities on disease-modifying therapy use in multiple sclerosis. Neurology 86, 1287–1295 (2016).

Issa, N. T., Byers, S. W. & Dakshanamurthy, S. Big data: the next frontier for innovation in therapeutics and healthcare. Expert Rev. Clin. Pharmacol. 7, 293–298 (2014).

Thorpe, K. E. et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J. Clin. Epidemiol. 62, 464–475 (2009).

van Staa, T. P. et al. The opportunities and challenges of pragmatic point-of-care randomised trials using routinely collected electronic records: evaluations of two exemplar trials. Health Technol. Assess. 18, 1–146 (2014).

Fiore, L. D. & Lavori, P. W. Integrating randomized comparative effectiveness research with patient care. N. Engl. J. Med. 374, 2152–2158 (2016).

Patsopoulos, N. A. A pragmatic view on pragmatic trials. Dialogues Clin. Neurosci. 13, 217–224 (2011).

Saaga, K. G. et al. Improving the efficiency and effectiveness of pragmatic clinical trials in older adults in the United States. Contemp. Clin. Trials 33, 1211–1216 (2012).