Development of machine learning models for the detection of surgical site infections following total hip and knee arthroplasty: a multicenter cohort study
Tóm tắt
Population based surveillance of surgical site infections (SSIs) requires precise case-finding strategies. We sought to develop and validate machine learning models to automate the process of complex (deep incisional/organ space) SSIs case detection. This retrospective cohort study included adult patients (age ≥ 18 years) admitted to Calgary, Canada acute care hospitals who underwent primary total elective hip (THA) or knee (TKA) arthroplasty between Jan 1st, 2013 and Aug 31st, 2020. True SSI conditions were judged by the Alberta Health Services Infection Prevention and Control (IPC) program staff. Using the IPC cases as labels, we developed and validated nine XGBoost models to identify deep incisional SSIs, organ space SSIs and complex SSIs using administrative data, electronic medical records (EMR) free text data, and both. The performance of machine learning models was assessed by sensitivity, specificity, positive predictive value, negative predictive value, F1 score, the area under the receiver operating characteristic curve (ROC AUC) and the area under the precision–recall curve (PR AUC). In addition, a bootstrap 95% confidence interval (95% CI) was calculated. There were 22,059 unique patients with 27,360 hospital admissions resulting in 88,351 days of hospital stay. This included 16,561 (60.5%) TKA and 10,799 (39.5%) THA procedures. There were 235 ascertained SSIs. Of them, 77 (32.8%) were superficial incisional SSIs, 57 (24.3%) were deep incisional SSIs, and 101 (42.9%) were organ space SSIs. The incidence rates were 0.37 for superficial incisional SSIs, 0.21 for deep incisional SSIs, 0.37 for organ space and 0.58 for complex SSIs per 100 surgical procedures, respectively. The optimal XGBoost models using administrative data and text data combined achieved a ROC AUC of 0.906 (95% CI 0.835–0.978), PR AUC of 0.637 (95% CI 0.528–0.746), and F1 score of 0.79 (0.67–0.90). Our findings suggest machine learning models derived from administrative data and EMR text data achieved high performance and can be used to automate the detection of complex SSIs.
Tài liệu tham khảo
Magill SS, Edwards JR, Bamberg W, et al. Multistate point-prevalence survey of health care–associated infections. N Engl J Med. 2014;370(13):1198–208.
Canadian Surgical Site Infection Prevention Audit Month February 2016. 2016:3. https://www.patientsafetyinstitute.ca/en/toolsResources/Documents/SSI%20Audit%202016_Recap%20Report%20EN.pdf
Adeyemi A, Trueman P. Economic burden of surgical site infections within the episode of care following joint replacement. J Orthop Surg Res. 2019;14(1):1–9.
Annual epidemiological report for 2018–2020. 2023. Healthcare-associated infections: surgical site infections. https://www.ecdc.europa.eu/sites/default/files/documents/Healthcare-associated%20infections%20-%20surgical%20site%20infections%202018-2020.pdf
Kurtz S, Ong K, Lau E, et al. Projections of primary and revision hip and knee arthroplasty in the United States from 2005 to 2030. JBJS. 2007;89(4):780–5.
Surgical Site Infection Event (SSI). 2023. https://www.cdc.gov/nhsn/pdfs/pscmanual/9pscssicurrent.pdf
Rennert-May E, Manns B, Smith S, et al. Validity of administrative data in identifying complex surgical site infections from a population-based cohort after primary hip and knee arthroplasty in Alberta, Canada. Am J Infect Control. 2018;46(10):1123–6.
Crocker A, Kornilo A, Conly J, et al. Using administrative data to determine rates of surgical site infections following spinal fusion and laminectomy procedures. Am J Infect Control. 2021;49(6):759–63.
Perdiz LB, Yokoe DS, Furtado GH, et al. Impact of an automated surveillance to detect surgical-site infections in patients undergoing total hip and knee arthroplasty in Brazil. Infect Control Hosp Epidemiol. 2016;37(8):991–3.
Quan H, Eastwood C, Cunningham CT, et al. Validity of AHRQ patient safety indicators derived from ICD-10 hospital discharge abstract data (chart review study). BMJ Open. 2013;3(10):e003716.
Wu G, Khair S, Yang F, et al. Performance of machine learning algorithms for surgical site infection case detection and prediction: a systematic review and meta-analysis. Ann Med Surg. 2022. https://doi.org/10.1016/j.amsu.2022.104956.
CCI Coding Structure. Canadian Institute for Health Information. 2023. https://www.cihi.ca/en/cci-coding-structure
Surgical site infection (SSI) prevention. 2017. https://www.health.gov.on.ca/en/pro/programs/ris/docs/ssi_prevention_en.pdf
Surgical Site Infection (SSI). 2023. https://www.cdc.gov/hai/ssi/ssi.html
Patient Safety Component Manual. National Healthcare Safety Network (NHSN). https://www.cdc.gov/nhsn/pdfs/pscmanual/pcsmanual_current.pdf
Keselj V. Speech and Language Processing Daniel Jurafsky and James H. Martin (Stanford University and University of Colorado at Boulder) Pearson Prentice Hall, 2009, xxxi+ 988 pp; hardbound, ISBN 978-0-13-187321-6, $115.00. MIT Press One Rogers Street, Cambridge, MA 02142–1209, USA journals-inf; 2009.
Chapman WW, Bridewell W, Hanbury P, et al. A simple algorithm for identifying negated findings and diseases in discharge summaries. J Biomed Inform. 2001;34(5):301–10.
Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825–30.
Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83.
Ozenne B, Subtil F, Maucort-Boulch D. The precision–recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases. J Clin Epidemiol. 2015;68(8):855–9.
vanRossum G. Python reference manual. Department of Computer Science [CS]. 1995; (R 9525)
Aiello FA, Shue B, Kini N, et al. Outcomes reported by the vascular quality initiative and the national surgical quality improvement program are not comparable. J Vascu Surg. 2014;60(1):152-159.e3.
HAI Progress Reports. Centers for Disease Control and Prevention. https://www.cdc.gov/nhsn/datastat/progress-report.html#anchor_1668529078141
Lethbridge LN, Richardson CG, Dunbar MJ. Measuring surgical site infection from linked administrative data following hip and knee replacement. J Arthroplast. 2020;35(2):528–33.
Urquhart DM, Hanna FS, Brennan SL, et al. Incidence and risk factors for deep surgical site infection after primary total hip arthroplasty: a systematic review. J Arthroplast. 2010;25(8):1216-1222.e3.
Ellison JJ, Boychuk LR, Chakravorty D, et al. A comparison of surgical site infections following total hip replacement and total knee replacement surgeries identified by infection prevention and control and the national surgical quality improvement program in alberta, Canada. Infect Control Hosp Epidemiol. 2022;43(4):435–41.
Surgical Site Infections following Total Hip and Total Knee Replacement (TH &TK SSIs) Protocol. 2022. https://www.albertahealthservices.ca/assets/healthinfo/ipc/hi-ipc-sr-hip-knee-ssi-protocol.pdf
Bucher BT, Shi J, Ferraro JP, et al. Portable automated surveillance of surgical site infections using natural language processing: development and validation. Ann Surg. 2020;272(4):629–36.
Rennert-May E, Bush K, Vickers D, et al. Use of a provincial surveillance system to characterize postoperative surgical site infections after primary hip and knee arthroplasty in Alberta, Canada. Amer J Infect Control. 2016;44(11):1310–4.