Elaborating on the assessment of the risk of bias in prognostic studies in pain rehabilitation using QUIPS—aspects of interrater agreement
Tóm tắt
Many studies have been performed to identify important prognostic factors for outcomes after rehabilitation of patients with chronic pain, and there is a need to synthesize them through systematic review. In this process, it is important to assess the study quality and risk of bias. The “Quality In Prognosis Studies” (QUIPS) tool has been developed for this purpose and consists of several prompting items categorized into six domains, and each domain is judged on a three-grade scale (low, moderate or high risk of bias). The aim of the present study was to determine the interrater agreement of the risk of bias assessment in prognostic studies of patients with chronic pain using QUIPS and to elaborate on the use of this instrument. We performed a systematic review and a meta-analysis of prognostic factors for long-term outcomes after multidisciplinary rehabilitation in patients with chronic pain. Two researchers rated the risk of bias in 43 published papers in two rounds (15 and 28 papers, respectively). The interrater agreement and Cohen’s quadratic weighted kappa coefficient (κ) and 95% confidence interval (95%CI) were calculated in all domains and separately for the first and second rounds. The raters agreed in 61% of the domains (157 out of 258), with similar interrater agreement in the first (59%, 53/90) and second rounds (62%, 104/168). The overall weighted kappa coefficient (kappa for all domains and all papers) was weak: κ = 0.475 (95%CI = 0.358–0.601). A “minimal agreement” between the raters was found in the first round, κ = 0.323 (95%CI = 0.129–0.517), but increased to “weak agreement” in the second round, κ = 0.536 (95%CI = 0.390–0.682). Despite a relatively low interrater agreement, QUIPS proved to be a useful tool in assessing the risk of bias when performing a meta-analysis of prognostic studies in pain rehabilitation, since it demands of raters to discuss and investigate important aspects of study quality. Some items were particularly hard to differentiate in-between, and a learning phase was required to increase the interrater agreement. This paper highlights several aspects of the tool that should be kept in mind when rating the risk of bias in prognostic studies, and provides some suggestions on common pitfalls to avoid during this process. PROSPERO
CRD42016025339
; registered 05 February 2016.
Tài liệu tham khảo
Armitage P, Gehan EA. Statistical methods for the identification and use of prognostic factors. Int J Cancer. 1974;13(1):16–36.
Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, Malats N, Briggs A, Schroter S, Altman DG, et al. Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013;10(2):e1001380.
Tseli E, Boersma K, Stålnacke B-M, Enthoven P, Gerdle B, Äng BO, Grooten WJA. Prognostic Factors for Physical Functioning After Multidisciplinary Rehabilitation in Patients With Chronic Musculoskeletal Pain: A Systematic Review and Meta-Analysis. Clin J Pain. 2019;35(2):148–73.
Mallen CD, Peat G, Thomas E, Dunn KM, Croft PR. Prognostic factors for musculoskeletal pain in primary care: a systematic review. Br J Gen Pract. 2007;57(541):655–61.
Croft P, Altman DG, Deeks JJ, Dunn KM, Hay AD, Hemingway H, LeResche L, Peat G, Perel P, Petersen SE, et al. The science of clinical practice: disease diagnosis or patient prognosis? Evidence about “what is likely to happen” should shape clinical practice. BMC Med. 2015;13:20.
Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, Briggs A, Udumyan R, Moons KGM, Steyerberg EW, et al. Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ. 2013;346:e5595.
Altman DG. Systematic reviews of evaluations of prognostic variables. BMJ. 2001;323(7306):224–8.
Altman DG, Riley RD. Primer: an evidence-based approach to prognostic markers. Nat Clin Pract Oncol. 2005;2(9):466–72.
Hemingway H, Riley RD, Altman DG. Ten steps towards improving prognosis research. BMJ. 2009;339:b4184.
Group CCPM: The Cochrane Collaboration Prognosis Methods Group, Review Tools. 2018.
Hayden JA, van Der Windt DA, Cartwright JL, Côté P, Bombardier C. Assessing bias in studies of prognostic factors. Ann Intern Med. 2013;158(4):280.
Tseli E, Grooten WJA, Stålnacke B-M, Boersma K, Enthoven P, Gerdle B, Äng BO. Predictors of multidisciplinary rehabilitation outcomes in patients with chronic musculoskeletal pain: protocol for a systematic review and meta-analysis. Syst Rev. 2017;6(1):199.
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22(3):276–82.
Chapple CM, Nicholson H, Baxter GD, Abbott JH. Patient characteristics that predict progression of knee osteoarthritis: a systematic review of prognostic studies. Arthritis Care Res. 2011;63(8):1115–25.
Elshout G, Monteny M, van der Wouden JC, Koes BW, Berger MY. Duration of fever and serious bacterial infections in children: a systematic review. BMC Fam Pract. 2011;12(1):33.
Gieteling MJ, Bierma-Zeinstra SM, Lisman-van Leeuwen Y, Passchier J, Berger MY. Prognostic factors for persistence of chronic abdominal pain in children. J Pediatr Gastroenterol Nutr. 2011;52(2):154–61.
Singh AS, Mulder C, Twisk JW, van Mechelen W, Chinapaw MJ. Tracking of childhood overweight into adulthood: a systematic review of the literature. Obes Rev. 2008;9(5):474–88.
Spee LA, Madderom MB, Pijpers M, van Leeuwen Y, Berger MY. Association between helicobacter pylori and gastrointestinal symptoms in children. Pediatrics. 2010;125(3):e651–69.
Jeejeebhoy FM, Zelop CM, Windrim R, Carvalho JC, Dorian P, Morrison LJ. Management of cardiac arrest in pregnancy: a systematic review. Resuscitation. 2011;82(7):801–9.
Johnson SR, Swinton JR, Granton JT. Prognostic factors for survival in scleroderma associated pulmonary arterial hypertension. J Rheumatol. 2008;35(8):1584–90.
Van Drongelen A, Boot CR, Merkus SL, Smid T, Van Der Beek AJ. The effects of shift work on body weight change—a systematic review of longitudinal studies. Scand J Work Environ Health. 2011;37:263–75.
Wright AA, Cook C, Abbott JH. Variables associated with the progression of hip osteoarthritis: a systematic review. Arthritis Care Res. 2009;61(7):925–36.
Kamper SJ, Rebbeck TJ, Maher CG, McAuley JH, Sterling M. Course and prognostic factors of whiplash: a systematic review and meta-analysis. Pain. 2008;138(3):617–29.
Bruls VE, Bastiaenen CH, de Bie RA. Prognostic factors of complaints of arm, neck, and/or shoulder: a systematic review of prospective cohort studies. Pain. 2015;156(5):765–88.
den Bakker CM, Anema JR, Zaman AG, de Vet HC, Sharp L, Angenete E, Allaix ME, Otten RH, Huirne JA, Bonjer HJ. Prognostic factors for return to work and work disability among colorectal cancer survivors; A systematic review. PLoS One. 2018;13(8):e0200720.
Bollen L, Jacobs W, Van der Linden Y, Van der Hel O, Taal W, Dijkstra P. A systematic review of prognostic factors predicting survival in patients with spinal bone metastases. Eur Spine J. 2018;27(4):799–805.
Paxton D, Pauling JD. Does nailfold capillaroscopy help predict future outcomes in systemic sclerosis? A systematic literature review. Seminars in Arthritis and Rheumatism. 2018;48(3):482–94.
Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85(3):257–68.