Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010

Intensive Care Medicine - 2013
Nicola Latronico1,2, Marta Metelli1, Maddalena Turin1, Simone Piva2, Frank A. Rasulo2, Cosetta Minelli3
1University Division of Anesthesia and Critical Care Medicine, Department of Anesthesia, Critical Care and Perioperative Medicine, University of Brescia, Brescia, Italy
2University Division of Neuroanesthesia and Neurocritical Care, Department of Anesthesia, Intensive Care and Perioperative Medicine, University of Brescia, Brescia, Italy
3Respiratory Epidemiology and Public Health Group, National Heart and Lung Institute, Imperial College London, London, UK

Tóm tắt

To evaluate the methodological quality of randomized controlled trials (RCTs) published in Intensive Care Medicine from 2001 to 2010, and to compare it with a previous review of RCTs published from 1975 to 2000. We assessed the quality of reporting of randomization, blinding and participant flow, both individually and combined within the Jadad scale, and compared them with findings from our previous review. For RCTs published from 2001 to 2010, we also evaluated the frequency of distorted finding presentation (spin) and inflated predicted treatment effect (delta inflation). In the 221 RCTs from 2001 to 2010, the sample size was significantly larger than in the older series, and there was a higher proportion of studies with negative findings. Reporting of the rationale for sample size estimation and allocation concealment increased significantly, but reporting of other important individual methodological components did not change substantially compared with the previous period and remained low. Among RCTs from 2001 to 2010, a spin strategy was used in 69 of 111 RCTs with statistically negative results, while delta inflation was present in 7 of 11 RCTs evaluating survival as a primary outcome. Papers with higher Jadad scores were cited more often than the others. Quality of reporting of RCTs published in Intensive Care Medicine has only partly improved over time, and spin and delta bias are of frequent occurrence. There is a need for stronger adherence to CONSORT recommendations, with special emphasis on accurate description of randomization and blindness, and correct reporting of statistically non-significant results.

Từ khóa


Tài liệu tham khảo

Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S (1995) Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 16:62–73

Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273:408–412

Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352:609–613

Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gotzsche PC, Lang T (2001) The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 134:663–694

Kjaergard LL, Villumsen J, Gluud C (2001) Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 135:982–989

Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S (2012) Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ 344:e1119

Savovic J, Jones HE, Altman DG, Harris RJ, Juni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, JP AI, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JA (2012) Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med

Jüni P, Altman DG, Egger M (2008) Assessing the quality of randomised controlled trialssystematic reviews in health care. BMJ, London, pp 87–108

Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ (1996) Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 17:1–12

Altman DG (2001) Systematic reviews of evaluations of prognostic variables. BMJ 323:224–228

Huwiler-Muntener K, Juni P, Junker C, Egger M (2002) Quality of reporting of randomized trials as a measure of methodologic quality. JAMA 287:2801–2804

Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA (2008) Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336:601–605

Aberegg SK, Richards DR, O’Brien JM (2010) Delta inflation: a bias in the design of randomized controlled trials in critical care medicine. Crit Care 14:R77

Boutron I, Dutton S, Ravaud P, Altman DG (2010) Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 303:2058–2064

Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG (2012) Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA 308:2594–2604

Latronico N, Botteri M, Minelli C, Zanotti C, Bertolini G, Candiani A (2002) Quality of reporting of randomised controlled trials in the intensive care literature. A systematic analysis of papers published in Intensive Care Medicine over 26 years. Intensive Care Med 28:1316–1323

Azoulay E, Citerio G, Timsit JF (2013) The identity of Intensive Care Medicine. Intensive Care Med 39:343–344

Sud S, Sud M, Friedrich JO, Meade MO, Ferguson ND, Wunsch H, Adhikari NK (2010) High frequency oscillation in patients with acute lung injury and acute respiratory distress syndrome (ARDS): systematic review and meta-analysis. BMJ 340:c2327

Felder TM, Palmer NR, Lal LS, Mullen PD (2011) What is the evidence for pharmaceutical patient assistance programs? A systematic review. J Health Care Poor Underserved 22:24–49

Poolman RW, Struijs PA, Krips R, Sierevelt IN, Lutz KH, Bhandari M (2006) Does a “Level I Evidence” rating imply high quality of reporting in orthopaedic randomised controlled trials? BMC Med Res Methodol 6:44

Lai TY, Wong VW, Lam RF, Cheng AC, Lam DS, Leung GM (2007) Quality of reporting of key methodological items of randomized controlled trials in clinical ophthalmic journals. Ophthalmic Epidemiol 14:390–398

Sut N, Senocak M, Uysal O, Koksalan H (2008) Assessing the quality of randomized controlled trials from two leading cancer journals using the CONSORT statement. Hematol Oncol Stem Cell Ther 1:38–43

Bai Y, Gao J, Zou DW, Li ZS (2009) Methodological reporting of randomized clinical trials in major gastroenterology and hepatology journals in 2006. Hepatology 49:2108–2112

Danilla S, Wasiak J, Searle S, Arriagada C, Pedreros C, Cleland H, Spinks A (2009) Methodological quality of randomised controlled trials in burns care. A systematic review. Burns 35:956–961

Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG (2010) The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ 340:c723

Strech D, Soltmann B, Weikert B, Bauer M, Pfennig A (2011) Quality of reporting of randomized controlled trials of pharmacologic treatment of bipolar disorders: a systematic review. J Clin Psychiatr 72:1214–1221

Agha RA, Camm CF, Edison E, Orgill DP (2012) The methodological quality of randomized controlled trials in plastic surgery needs improvement: a systematic review. J Plast Reconstr Aesthet Surg

Mills EJ, Wu P, Gagnier J, Devereaux PJ (2005) The quality of randomized trial reporting in leading medical journals since the revised CONSORT statement. Contemp Clin Trials 26:480–487

Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, Dias S, Schulz KF, Plint AC, Moher D (2012) Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev 11:MR000030

Schulz KF, Altman DG, Moher D (2010) CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med 152:726–732

Dechartres A, Charles P, Hopewell S, Ravaud P, Altman DG (2011) Reviews assessing the quality or the reporting of randomized controlled trials are increasing over time but raised questions about how quality is assessed. J Clin Epidemiol 64:136–144

Hirst A, Altman DG (2012) Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PLoS ONE 7:e35621

Karanicolas PJ, Farrokhyar F, Bhandari M (2010) Practical tips for surgical research: blinding: who, what, when, why, how? Can J Surg 53:345–348

Schulz KF, Altman DG, Moher D, Fergusson D (2010) CONSORT 2010 changes and testing blindness in RCTs. Lancet 375:1144–1146

Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P (2009) Reporting of sample size calculation in randomised controlled trials: review. BMJ 338:b1732

Moher D, Fortin P, Jadad AR, Juni P, Klassen T, Le Lorier J, Liberati A, Linde K, Penna A (1996) Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews. Lancet 347:363–366

Schulz KF, Grimes DA (2002) Allocation concealment in randomised trials: defending against deciphering. Lancet 359:614–618

Guyatt GH, Mills EJ, Elbourne D (2008) In the era of systematic reviews, does the size of an individual trial still matter. PLoS Med 5:e4

Schulz KF, Grimes DA (2005) Multiplicity in randomised trials II: subgroup and interim analyses. Lancet 365:1657–1661

Schulz KF, Grimes DA (2005) Multiplicity in randomised trials I: endpoints and treatments. Lancet 365:1591–1595

Schulz KF, Grimes DA (2005) Sample size calculations in randomised trials: mandatory and mystical. Lancet 365:1348–1353

Lundh A, Gotzsche PC (2008) Recommendations by Cochrane Review Groups for assessment of the risk of bias in studies. BMC Med Res Methodol 8:22

Hyman M (2010) Science for sale: protect yourself from medical research deception. http://www.huffingtonpost.com/dr-mark-hyman/dangerous-spin-doctors-7-_b_747325.html. Accessed 28 June 2012

International Committee of Medical Journal Editors (2013) Uniform requirements for manuscripts submitted to biomedical journals: publishing and editorial issues related to publication in biomedical journals: obligation to publish negative studies. http://www.icmje.org/publishing_1negative.html. Accessed 2 April 2013

Altman DG, Bland JM (1995) Absence of evidence is not evidence of absence. BMJ 311:485

Powers JH (2008) Noninferiority and equivalence trials: deciphering ‘similarity’ of medical interventions. Stat Med 27:343–352

Drazen JM (2012) Believe the data. N Engl J Med 367:1152–1153

Kesselheim AS, Robertson CT, Myers JA, Rose SL, Gillet VBA, Ross KM, Glynn RJ, Joffe S, Avorn J (2012) A randomized study of how physicians interpret research funding disclosures. N Engl J Med 367:1119–1127

(2005) In praise of soft science. Nature 435:1003

Bonati MR, Drusini AG (1996) Morgagni and the impact factor. Nature 381:271–271

Smith R (2003) Medical journals and pharmaceutical companies: uneasy bedfellows. BMJ 326:1202–1205

Smith R (2005) Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med 2:e138

Lundh A, Barbateskovic M, Hrobjartsson A, Gotzsche PC (2010) Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue—cohort study. PLoS Med 7:e1000354

McVeigh ME, Mann SJ (2009) The journal impact factor denominator: defining citable (counted) items. JAMA 302:1107–1109

Callaham M, Wears RL, Weber E (2002) Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA 287:2847–2850

Nieminen P, Carpenter J, Rucker G, Schumacher M (2006) The relationship between quality of research and citation frequency. BMC Med Res Methodol 6:42

Etter JF, Stapleton J (2009) Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. J Clin Epidemiol 62:831–837

Filion KB, Pless IB (2008) Factors related to the frequency of citation of epidemiologic publications. Epidemiol Perspect Innov EP+I 5:3

Thông tin
Thông tin xuất bản

Intensive Care Medicine

Thông tin tác giả