A machine learning approach for modeling decisions in the out of hospital cardiac arrest care workflow
Tóm tắt
A growing body of research has shown that machine learning (ML) can be a useful tool to predict how different variable combinations affect out-of-hospital cardiac arrest (OHCA) survival outcomes. However, there remain significant research gaps on the utilization of ML models for decision-making and their impact on survival outcomes. The purpose of this study was to develop ML models that effectively predict hospital’s practice to perform coronary angiography (CA) in adult patients after OHCA and subsequent neurologic outcomes. We utilized all (N = 2398) patients treated by the Chicago Fire Department Emergency Medical Services included in the Cardiac Arrest Registry to Enhance Survival (CARES) between 2013 and 2018 who survived to hospital admission to develop, test, and analyze ML models for decisions after return of spontaneous circulation (ROSC) and patient survival. ML classification models, including the Embedded Fully Convolutional Network (EFCN) model, were compared based on their ability to predict post-ROSC decisions and survival. The EFCN classification model achieved the best results across tested ML algorithms. The area under the receiver operating characteristic curve (AUROC) for CA and Survival were 0.908 and 0.896 respectively. Through cohort analyses, our model predicts that 18.3% (CI 16.4–20.2) of patients should receive a CA that did not originally, and 30.1% (CI 28.5–31.7) of these would experience improved survival outcomes. ML modeling effectively predicted hospital decisions and neurologic outcomes. ML modeling may serve as a quality improvement tool to inform system level OHCA policies and treatment protocols.
Tài liệu tham khảo
Daya MR, Schmicker RH, Zive DM, et al. Out-of-hospital cardiac arrest survival improving over time: results from the resuscitation outcomes consortium (ROC). Resuscitation. 2015;91:108–15.
Govindarajan P, Lin L, Landman A, McMullan JT, McNally BF, Crouch AJ, Sasson C. Practice variability among the EMS systems participating in cardiac arrest registry to enhance survival (CARES). Resuscitation. 2012;83(1):76–80.
Coppler PJ, Sawyer KN, Youn CS, et al. Variability of post-cardiac arrest care practices among cardiac arrest centers: United States and South Korean dual network survey of emergency physician research principal investigators. Ther Hypothermia Temp Manag. 2017;7(1):30–5. https://doi.org/10.1089/ther.2016.0017.
Wiens J, Shenoy ES. Machine learning for healthcare: on the verge of a major shift in healthcare epidemiology. Clin Infect Dis. 2018;66(1):149–53.
Ngiam KY, Khor IW. Big data and machine learning algorithms for health-care delivery. Lancet Oncol. 2019;20(5):e262–73.
Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):1317–8.
Giger ML. Machine learning in medical imaging. J Am College Radiol. 2018;15(3):512–20.
Ardabili SF, Mosavi A, Ghamisi P, Ferdinand F, Varkonyi-Koczy AR, Reuter U, Rabczuk T, Atkinson PM. Covid-19 outbreak prediction with machine learning. Available at SSRN 3580188. 2020.
Vamathevan J, Clark D, Czodrowski P, Dunham I, Ferran E, Lee G, Li B, Madabhushi A, Shah P, Spitzer M, Zhao S. Applications of machine learning in drug discovery and development. Nat Rev Drug Discov. 2019;18(6):463–77.
Sharabiani A, Darabi H, Bress A, Cavallari L, Nutescu E, Drozda K. Machine learning based prediction of warfarin optimal dosing for African American patients. In: 2013 IEEE international conference on automation science and engineering (CASE) 2013; pp 623–628. IEEE.
Darabi H, Galanter WL, Lin JY, Buy U, Sampath R. Modeling and integration of hospital information systems with Petri nets. In: 2009 IEEE/INFORMS international conference on service operations, logistics and informatics 2009, pp 190–195. IEEE.
Haji M, Darabi H. A simulation case study: Reducing outpatient waiting time of otolaryngology care services using VBA. In: 2011 IEEE international conference on automation science and engineering 2011, pp 525–530. IEEE.
Blomberg SN, Folke F, Ersbøll AK, Christensen HC, Torp-Pedersen C, Sayre MR, Counts CR, Lippert FK. Machine learning as a supportive tool to recognize cardiac arrest in emergency calls. Resuscitation. 2019;138:322–9.
Majumder AK, ElSaadany YA, Young R, Ucci DR. An energy efficient wearable smart IoT system to predict cardiac arrest. Adv Hum-Comput Interact. 2019. https://doi.org/10.1155/2019/1507465.
Kwon JM, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. J Am Heart Assoc. 2018;7(13):e008678.
Krizmaric M, Verlic M, Stiglic G, Grmec S, Kokol P. Intelligent analysis in predicting outcome of out-of-hospital cardiac arrest. Comput Methods Progr Biomed. 2009;95(2 Suppl):S22-32.
Nanayakkara S, Fogarty S, Tremeer M, et al. Characterising risk of in-hospital mortality following cardiac arrest using machine learning: a retrospective international registry study. PLoS Med. 2018;15(11):e1002709.
Region 11 Chicago EMS—Region 11 Chicago EMS. 21 September 2020. https://chicagoems.org.
REGION XI CHICAGO EMS SYSTEM POLICIES AND PROCEDURES, 2017, https://chicagoems.org/wp-content/uploads/sites/2/2017/08/2017-PP_APPROVED.pdf.
MyCares. 21 September 2020, https://mycares.net/
McNally B, Stokes A, Crouch A, Kellermann AL, CARES Surveillance Group. CARES: cardiac arrest registry to enhance survival. Ann Emerg Med. 2009;54(5):674–83.
Potdar K, Pardawala TS, Pai CD. A comparative study of categorical variable encoding techniques for neural network classifiers. Int J Comput Appl. 2017;175(4):7–9.
Reynolds JC, Callaway CW, El Khoudary SR, Moore CG, Alvarez RJ, Rittenberger JC. Coronary angiography predicts improved outcome following cardiac arrest: propensity-adjusted analysis. J Intensive Care Med. 2009;24:179–86.
Dumas F, Cariou A, Manzo-Silberman S, Grimaldi D, Vivien B, Rosencher J, Empana JP, Carli P, Mira JP, Jouven X, Spaulding C. Immediate percutaneous coronary intervention is associated with better survival after out-of-hospital cardiac arrest: insights from the PROCAT (Parisian Region Out of hospital Cardiac ArresT) registry. Circul: Cardiovasc Interv. 2010;3(3):200–7.
Hollenbeck RD, McPherson JA, Mooney MR, Unger BT, Patel NC, McMullan PW Jr, Hsu CH, Seder DB, Kern KB. Early cardiac catheterization is associated with improved survival in comatose survivors of cardiac arrest without STEMI. Resuscitation. 2014;85:88–95.
Grossestreuer AV, Abella BS, Sheak KR, et al. Inter-rater reliability of post-arrest cerebral performance category (CPC) scores. Resuscitation. 2016;109:21–4.
Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY. Lightgbm: A highly efficient gradient boosting decision tree. In: Advances in neural information processing systems 2017, pp 3146–3154.
Chen T, Guestrin C. Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining 2016, pp 785–794.
Safavian SR, Landgrebe D. A survey of decision tree classifier methodology. IEEE Trans Syst Man Cybern. 1991;21(3):660–74.
Liaw A, Wiener M. Classification and regression by random Forest. R news. 2002;2(3):18–22.
Peter S, Diego F, Hamprecht FA, Nadler B. Cost efficient gradient boosting. In: Advances in neural information processing systems 2017, pp 1551–1561.
Peterson LE. K-nearest neighbor. Scholarpedia. 2009;4(2):1883.
Hosmer DW Jr, Lemeshow S, Sturdivant RX. Applied logistic regression. Hoboken: Wiley; 2013.
Yu H, Kim S. SVM tutorial-classification, regression and ranking. Handb Natural Comput. 2012;1:479–506.
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.
Harford S, Darabi H, Del Rios M, Majumdar S, Karim F, Hoek TV, Erwin K, Watson DP. A machine learning based model for out of hospital cardiac arrest outcome classification and sensitivity analysis. Resuscitation. 2019;138:134–40.
Jia Y, Zhang Y, Weiss R, Wang Q, Shen J, Ren F, Nguyen P, Pang R, Moreno IL, Wu Y. Transfer learning from speaker verification to multispeaker text-to-speech synthesis. In: Advances in neural information processing systems 2018, pp 4480–4490.
Chicco D, Jurman G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics. 2020;21(1):1–3.
Roulston MS. Performance targets and the Brier score. Meteorol Appl: J Forecast Pract Appl Train Tech Model. 2007;14(2):185–94.
Sundararajan M, Najmi A. The many Shapley values for model explanation. In: International conference on machine learning 2020, pp 9269–9278. PMLR.
Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. 2019;380(14):1347–58.
Al-Dury N, Ravn-Fischer A, Hollenberg J, Israelsson J, Nordberg P, Strömsöe A, Axelsson C, Herlitz J, Rawshani A. Identifying the relative importance of predictors of survival in out of hospital cardiac arrest: a machine learning study. Scand J Trauma Resusc Emerg Med. 2020;28(1):1–8.
Kwon JM, Jeon KH, Kim HM, Kim MJ, Lim S, Kim KH, Song PS, Park J, Choi RK, Oh BH. Deep-learning-based out-of-hospital cardiac arrest prognostic system to predict clinical outcomes. Resuscitation. 2019;139:84–91. https://doi.org/10.1016/j.resuscitation.2019.04.007.
Callaway CW, Schmicker R, Kampmeyer M, et al. Receiving hospital characteristics associated with survival after out-of-hospital cardiac arrest. Resuscitation. 2010;81(5):524–9.
Carr BG, Kahn JM, Merchant RM, et al. Inter-hospital variability in post-cardiac arrest mortality. Resuscitation. 2009;80(1):30–4.
Schober A, Sterz F, Laggner AN, et al. Admission of out-of-hospital cardiac arrest victims to a high volume cardiac arrest center is linked to improved outcome. Resuscitation. 2016;106:42–8.
Blum N, Del Rios M, Kotini P, Nguyen H, Campbell T, Markul E, Weber J, Vanden HT. Interhospital variability in out-of-hospital cardiac arrest treatment and survival in a large metropolitan Aea. Acad Emerg Med. 2019;26(S1):A353.