Interpretation and Impact of Real-World Clinical Data for the Practicing Clinician

Luce BR, Drummond M, Jönsson B, et al. EBM, HTA, and CER: clearing the confusion. Milbank Q. 2010;88:256–76.

Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence—what is it and what can it tell us? N Engl J Med. 2016;375:2293–7.

Barnish MS, Turner S. The value of pragmatic and observational studies in health care and public health. Pragmat Obs Res. 2017;8:49–55.

Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L. Randomized controlled trials: do they have external validity for patients with multiple comorbidities? Ann Fam Med. 2006;4(2):104–8.

FDA. Developing a framework for regulatory use of real-world evidence; Public Workshop. https://www.gpo.gov/fdsys/pkg/FR-2017-07-31/pdf/2017-16021.pdf . Accessed 08 Sep 2017.

EMA. Update on real world evidence data collection. 10 March 2016. https://ec.europa.eu/health//sites/health/files/files/committee/stamp/2016-03_ stamp4/4_ real_world_evidence_ema_presentation.pdf. Accessed 08 Sep 2017.

Batrouni M, Comet D, Meunier JP. Real world studies, challenges, needs and trends from the industry. Value Health. 2014;17:A587–8.

Goodman CS. National Information Center on Health Services Research and Health Care Technology (NICHSR): HTA 101, 2017. https://www.nlm.nih.gov/nichsr/hta101/ta10103.html . Accessed Feb 2018.

Malone DC, Brown M, Hurwitz JT, Peters L, Graff JS. Real-world evidence: useful in the real world of US payer decision making? How? When? And what studies? Value Health. 2018;21(3):326–33.

Cattell J, Groves P, Hughes B, Savas S. How can pharmacos take advantage of the real-world data opportunity in healthcare? McKinsey and Company, 2011. https://www.mckinsey.com/~/media/mckinsey/dotcom/client_service/Pharma%20and%20Medical%20Products/PMP%20NEW/PDFs/Pharma%20%20RWD%20opportunity%20October%202011.ashx . Accessed Feb 2018.

ABPI. The vision for real world data—harnessing the opportunities in the UK. Demonstrating value with real world data 2017. http://www.abpi.org.uk/media/1378/vision-for-real-world-data.pdf . Accessed 22 Jan 2018.

Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Clin Epidemiol. 2009;62(5):499–505.

Davies J, Martinex M, Martina R, et al. Retrospective indirect comparison of alectinib phase II data vs ceritinib real-world data in ALK + NSCLC after progression on crizotinib. Ann Oncol. 2017;28(suppl_2): ii28-ii51. 10.

Ford I, Norrie J. Pragmatic trials. N Engl J Med. 2016;375:454–63.

Dang A, Vallish BN. Real world evidence: an Indian perspective. Perspect Clin Res. 2016;7:156–60.

Sox HC, Lewis RJ. Pragmatic trials: practical answers to “real world” questions. JAMA. 2016;316:1205–6.

Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290:1624–32.

Dubois RW. Is the real-world evidence or hypothesis: a tale of two retrospective studies. J Comp Eff Res. 2015;4(3):199–201.

Clinicaltrials.gov. Studies for “Diabetes Mellitus, Type 2”. https://clinicaltrials.gov/ct2/results?cond=Diabetes+Mellitus%2C+Type+2&term=&cntry=&state=&city=&dist= . Accessed 21 Aug 2018.

Carls GS, Tuttle E, Tan RD, et al. Understanding the gap between efficacy in randomized controlled trials and effectiveness in real-world use of GLP-1 RA and DPP-4 therapies in patients with type 2 diabetes. Diabetes Care. 2017;40:1469–78.

Edelman SV, Polonsky WH. Type 2 diabetes in the real world: the elusive nature of glycemic control. Diabetes Care. 2017;40:1425–32.

McGovern A, Hinchliffe R, Munro N, de Lusignan S. Basing approval of drugs for type 2 diabetes on real world outcomes. BMJ. 2015;351:h5829.

Zhou FL, Ye F, Gupta V, et al. Older adults with type 2 diabetes (T2D) experience less hypoglycemia when switching to insulin glargine 300 U/mL (Gla-300) vs other basal insulins (DELIVER 3 study). Poster 986-P, American Diabetes Association (ADA) 77th Scientific Sessions, San Diego, CA, US, June 10, 2017.

Blonde L, Merilainen M, Karwe V, Raskin P. Patient-directed titration for achieving glycaemic goals using a once-daily basal insulin analogue: an assessment of two different fasting plasma glucose targets-the TITRATE™ study. Diabetes Obes Metab. 2009;11:623–31.

Gerstein HC, Yale JF, Harris SB, et al. A randomized trial of adding insulin glargine vs. avoidance of insulin in people with type 2 diabetes on either no oral glucose-lowering agents or submaximal doses of metformin and/or sulphonylureas. The Canadian INSIGHT (Implementing New Strategies with Insulin Glargine for Hyperglycaemia Treatment) Study. Diabet Med. 2006;23:736–42.

Meneghini L, Koenen C, Weng W, Selam JL. The usage of a simplified self-titration dosing guideline (303 Algorithm) for insulin detemir in patients with type 2 diabetes—results of the randomized, controlled PREDICTIVE™ 303 study. Diabetes Obes Metab. 2007;9:902–13.

Garrison LP Jr, Neumann PJ, Erickson P, Marshall D, Mullins CD. Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. Value Health. 2007;10:326–35.

Roche N, Reddel H, Martin R, et al. Quality standards for real-world research. Focus on observational database studies of comparative effectiveness. Ann Am Thorac Soc. 2014;11(Suppl 2):S99–104.

Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med. 2000;22(342):1878–86.

Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342:1887–92.

Golder S, Loke YK, Bland M. Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med. 2011;8:e1001026.

McMurry TL, Hu Y, Blackstone EH, Kozower BD. Propensity scores: methods, considerations, and applications. J Thorac Cardiovasc Surg. 2015;150:14–9.

Patsopoulos NA. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011;13:217–24.

Frieden TR. Evidence for health decision making—beyond randomized, controlled trials. N Engl J Med. 2017;377:465–75.

Ye F, Agarwal R, Kaur A, et al. Real-world assessment of patient characteristics and clinical outcomes of early users of the new insulin glargine 300U/mL. Poster 943-P, American Diabetes Association (ADA) 76th Scientific Sessions, New Orleans, LA, US. June 11, 2016.

Zhou FL, Ye F, Berhanu P, et al. Real-world evidence concerning clinical and economic outcomes of switching to insulin glargine 300 units/mL vs other basal insulins in patients with type 2 diabetes using basal insulin. Diabetes Obes Metab. 2018;20(5):1293–7.

Bolli GB, Riddle MC, Bergenstal RM, et al. New insulin glargine 300 U/ml compared with glargine 100 U/ml in insulin-naïve people with type 2 diabetes on oral glucose-lowering drugs: a randomized controlled trial (EDITION 3). Diabetes Obes Metab. 2015;17:386–94.

Riddle MC, Bolli GB, Ziemen M, et al. New insulin glargine 300 units/mL versus glargine 100 units/mL in people with type 2 diabetes using basal and mealtime insulin: glucose control and hypoglycemia in a 6-month randomized controlled trial (EDITION 1). Diabetes Care. 2014;37:2755–62.

Yki-Järvinen H, Bergenstal R, Ziemen M, et al. New insulin glargine 300 units/mL versus glargine 100 units/mL in people with type 2 diabetes using oral agents and basal insulin: glucose control and hypoglycemia in a 6-month randomized controlled trial (EDITION 2). Diabetes Care. 2014;37:3235–43.

Mahmood SS, Levy D, Vasan RS, Wang TJ. The Framingham Heart Study and the epidemiology of cardiovascular disease: a historical perspective. Lancet. 2014;383:999–1008.

Stratton IM, Adler AI, Neil HA, et al. Association with glycaemia with macrovascular and microvascular complications of type 2 diabetes (UKPDS 35): prospective observational study. BMJ. 2000;321:405–12.

Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications Research Group. Effect of intensive therapy on the microvascular complications of type 1 diabetes mellitus. JAMA. 2002;287:2563–9.

Freemantle N, Danchin N, Calvi-Gries F, Vincent M, Home PD. Relationship of glycaemic control and hypoglycaemic episodes to 4-year cardiovascular outcomes in people with type 2 diabetes starting insulin. Diabetes Obes Metab. 2016;18:152–8.

Nathan DM, Buse JB, Kahn SE, et al. Rationale and design of the glycemia reduction approaches in diabetes: a comparative effectiveness study (GRADE). Diabetes Care. 2013;36:2254–61.

Wermeling PR, Gorter KJ, Stellato RK, et al. Effectiveness and cost-effectiveness of 3-monthly versus 6-monthly monitoring of well-controlled type 2 diabetes patients: a pragmatic randomised controlled patient-preference equivalence trial in primary care (EFFIMODI study). Diabetes Obes Metab. 2014;16:841–9. https://doi.org/10.1111/dom.12288 .

Young LA, Buse JB, Weaver MA, et al. Three approaches to glucose monitoring in non-insulin treated diabetes: a pragmatic randomized clinical trial protocol. BMC Health Serv Res. 2017;17:369.

Furler J, O’Neal D, Speight J, et al. Supporting insulin initiation in type 2 diabetes in primary care: results of the Stepping Up pragmatic cluster randomised controlled clinical trial. BMJ. 2017;356:j783.

Choudhry NK, Isaac T, Lauffenburger JC, et al. Rationale and design of the Study of a Tele-pharmacy Intervention for Chronic diseases to Improve Treatment adherence (STIC2IT): a cluster-randomized pragmatic trial. Am Heart J. 2016;180:90–7.

Green JB, Bethel MA, Armstrong PW, et al. Effect of sitagliptin on cardiovascular outcomes in type 2 diabetes. N Engl J Med. 2015;373:232–42.

Holman RR, Bethel MA, Mentz RJ, et al. Effects of once-weekly exenatide on cardiovascular outcomes in type 2 diabetes. N Engl J Med. 2017;377:1228–39.

Zinman B, Wanner C, Lachin JM, et al. Empagliflozin, cardiovascular outcomes, and mortality in type 2 diabetes. N Engl J Med. 2015;373:2117–28.

Neal B, Perkovic V, Mahaffey KW, et al. Canagliflozin and cardiovascular and renal events in type 2 diabetes. N Engl J Med. 2017;377(7):644–57.

Kosiborod M, Cavender MA, Fu AZ, et al. Lower risk of heart failure and death in patients initiated on sodium-glucose cotransporter-2 inhibitors versus other glucose-lowering drugs: the CVD-REAL study (comparative effectiveness of cardiovascular outcomes in new users of sodium-glucose cotransporter-2 inhibitors). Circulation. 2017;136(3):249–59.

STROBE. STROBE Statement: Strengthening the reporting of observational studies in epidemiology. https://www.strobe-statement.org/index.php?id=available-checklists . Accessed 26 Sep 2018.

Zwarenstein M, Treweek S, Gagnier JJ, et al. Pragmatic Trials in Healthcare (Practihc) group. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:a2390.

Pitt B, Zannad F, Remme WJ, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med. 1999;341(10):709–17.

Freemantle N, Marston L, Walters K, et al. Making inferences on treatment effects from real world data: propensity scores, confounding by indication, and other perils for the unwary in observational research. BMJ. 2013;347:f6409.

Penning de Vries BBL, Groenwold RHH. Cautionary note: propensity score matching does not account for bias due to censoring. Nephrol Dial Transplant. 2017;1–3.

Zhang R, Wang Y, Liu B, et al. Clinical data quality problems and countermeasure for real world study. Front Med. 2014;8:352–57.

Chen JH, Asch SM. Machine learning and prediction in medicine—beyond the peak of inflated expectations. N Engl J Med. 2017;376(26):2507.