Newcastle-Ottawa Scale: comparing reviewers’ to authors’ assessments

Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. Edited by: Higgins JPT, Green S. [http://www.cochrane-handbook.org]

Wells G, Shea B, O’Connell D, Peterson J, Welch V, Losos M, Tugwell P: The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2013, http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp,

Hartling L, Milne A, Hamm MP, Vandermeer B, Ansari M, Tsertsvadze A, Dryden DM: Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers. J Clin Epidemiol. 2013, 66: 982-993. 10.1016/j.jclinepi.2013.03.003.

Oremus M, Oremus C, Hall GBC, McKinnon MC, ECT & Cognition Systematic Review Team: Inter-rater and test–retest reliability of quality assessments by novice student raters using the Jadad and Newcastle–Ottawa Scales. BMJ Open. 2012, 2: e001368-

Devereaux PJ, Choi PTL, El-Dika S, Bhandari M, Montori VM, Schünemann HJ, Garg AX, Busse JW, Heels-Ansdell D, Ghali WA, Manns BJ, Guyatt GH: An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol. 2004, 57: 1232-1236. 10.1016/j.jclinepi.2004.03.017.

Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, Djulbegovic B: Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group. BMJ. 2004, 328: 22-24. 10.1136/bmj.328.7430.22.

Mertz D, Kim TH, Johnstone J, Lam P-P, Science M, Kuster SP, Fadel SA, Tran D, Fernandez E, Bhatnagar N, Loeb M: Populations at risk for severe or complicated influenza illness: a systematic review and meta-analysis. BMJ. 2012, 347: f5061-

Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174. 10.2307/2529310.

Ben-David A: Comparison of classification accuracy using Cohen’s Weighted Kappa. Expert Syst Appl. 2008, 34: 825-832. 10.1016/j.eswa.2006.10.022.

Mahnken A, Koos R, Katoh M, Spuentrup E, Busch P, Wildberger J, Kühl H, Günther R: Sixteen-slice spiral CT versus MR imaging for the assessment of left ventricular function in acute myocardial infarction. Eur Radiol. 2005, 15: 714-720. 10.1007/s00330-004-2592-x.

Wells G, Brodsky L, O’Connell D, Shea B, Henry D, Mayank S, Tugwell P: Evaluation of the Newcastle-Ottawa Scale (NOS): an assessment tool for evaluating the quality of non-randomized studies. XI Cochrane Colloquium: Evidence, Health Care and Culture. 2003, Barcelona, Spain, http://www.citeulike.org/user/SRCMethodsLibrary/article/12919189,

Strijbos J-W, Martens RL, Prins FJ, Jochems WMG: Content analysis: what are they talking about?. Comput Educ. 2006, 46: 29-48. 10.1016/j.compedu.2005.04.002.