Prospective External Validation of a Commercial Model Predicting the Likelihood of Inpatient Admission From the Emergency Department

Forster, 2003, The effect of hospital occupancy on emergency department length of stay and patient disposition, Acad Emerg Med, 10, 127, 10.1197/aemj.10.2.127 Richardson, 2009, Myths versus facts in emergency department overcrowding and hospital access block, Med J Aust, 190, 369, 10.5694/j.1326-5377.2009.tb02451.x Kelen GD, Wolfe R, D’Onofrio G, et al. Emergency department crowding: the canary in the health care system. NEJM Catalyst. Published online September 28, 2021. Bernstein, 2009, The effect of emergency department crowding on clinically oriented outcomes, Acad Emerg Med, 16, 1, 10.1111/j.1553-2712.2008.00295.x Stoyanov, 2020, Effects of crowding in the emergency department on the diagnosis and management of suspected acute coronary syndrome using rapid algorithms: an observational study, BMJ Open, 10, 10.1136/bmjopen-2020-041757 Hoot, 2008, Systematic review of emergency department crowding: causes, effects, and solutions, Ann of Emerg Med, 52, 126, 10.1016/j.annemergmed.2008.03.014 Hobbs, 2000, Hospital factors associated with emergency center patients leaving without being seen, Am J Emerg Med, 18, 767, 10.1053/ajem.2000.18075 Gorski, 2021, Crowding is the strongest predictor of left without being seen risk in a pediatric emergency department, Am J Emerg Med, 48, 73, 10.1016/j.ajem.2021.04.005 Bursch, 1993, Emergency department satisfaction: what matters most?, Ann Emerg Med, 22, 586, 10.1016/S0196-0644(05)81947-X Schull, 2004, Emergency department crowding and thrombolysis delays in acute myocardial infarction, Ann Emerg Med, 44, 577, 10.1016/j.annemergmed.2004.05.004 Lee, 2020, Using emergency physicians’ abilities to predict patient admission to decrease admission delay time, Emerg Med J, 37, 417, 10.1136/emermed-2019-208859 Peck, 2014, Characterizing the value of predictive analytics in facilitating hospital patient flow, IIE Trans Healthc Syst Eng, 4, 135, 10.1080/19488300.2014.930765 Qiu, 2015, A cost sensitive inpatient bed reservation approach to reduce emergency department boarding times, Health Care Manag Sci, 18, 67, 10.1007/s10729-014-9283-1 Resar, 2011, Using real-time demand capacity management to improve hospitalwide patient flow, Jt Comm J Qual Patient Saf, 37 Cameron, 2015, A simple tool to predict admission at the time of triage, Emerg Med J, 32, 174, 10.1136/emermed-2013-203200 Sun, 2011, Predicting hospital admissions at emergency department triage using routine administrative data, Acad Emerg Med, 18, 844, 10.1111/j.1553-2712.2011.01125.x Parker, 2019, Predicting hospital admission at the emergency department triage: a novel prediction model, Am J Emerg Med, 37, 1498, 10.1016/j.ajem.2018.10.060 Peck, 2012, Predicting emergency department inpatient admissions to improve same-day patient flow, Acad Emerg Med, 19, E1045, 10.1111/j.1553-2712.2012.01435.x Hong, 2018, Predicting hospital admission at emergency department triage using machine learning, PloS One, 13, 10.1371/journal.pone.0201016 Barak-Corren, 2017, Progressive prediction of hospitalisation in the emergency department: uncovering hidden patterns to improve patient flow, Emerg Med J, 34, 308, 10.1136/emermed-2014-203819 Lucke, 2018, Early prediction of hospital admission for emergency department patients: a comparison between patients younger or older than 70 years, Emerg Med J, 35, 18, 10.1136/emermed-2016-205846 Brink, 2022, Predicting inhospital admission at the emergency department: a systematic review, Emerg Med J, 39, 191, 10.1136/emermed-2020-210902 Kraaijvanger, 2018, Development and validation of an admission prediction tool for emergency departments in the Netherlands, Emerg Med J, 35, 464, 10.1136/emermed-2017-206673 Fenn, 2021, Development and validation of machine learning models to predict admission from emergency department to inpatient and intensive care units, Ann Emerg Med, 78, 290, 10.1016/j.annemergmed.2021.02.029 King, 2022, Machine learning for real-time aggregated prediction of hospital admission for emergency patients, NPJ Digital Med, 5, 104, 10.1038/s41746-022-00649-y Cabitza, 2021, The importance of being external. methodological insights for the external validation of machine learning models in medicine, Comput Methods Programs Biomed, 208, 10.1016/j.cmpb.2021.106288 Beam, 2020, Challenges to the reproducibility of machine learning models in health care, JAMA, 323, 305, 10.1001/jama.2019.20866 Calster, 2019, Predictive analytics in health care: how can we know it works?, J Am Med Inform Assoc, 26, 1651, 10.1093/jamia/ocz130 Blauer Singh, 2021, Evaluating a widely implemented proprietary deterioration index model among hospitalized patients with COVID-19, Ann Am Thorac Soc, 18, 1129, 10.1513/AnnalsATS.202006-698OC Wong, 2021, External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients, JAMA Intern Med, 181, 1065, 10.1001/jamainternmed.2021.2626 Moons, 2015, Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration, Ann Intern Med, 162, W1, 10.7326/M14-0698 Riley, 2021, Minimum sample size for external validation of a clinical prediction model with a binary outcome, Stat Med, 40, 4230, 10.1002/sim.9025 Saito, 2015, The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets, PLoS One, 10bh Huang, 2020, A tutorial on calibration measurements and calibration models for clinical prediction models, J Am Med Inform Assoc, 27, 621, 10.1093/jamia/ocz228 Allen, 2019, Raincloud plots: a multi-platform tool for robust data visualization, Wellcome Open Res, 4, 63, 10.12688/wellcomeopenres.15191.1 Wickham, 2019, Welcome to the Tidyverse, J Open Source Softw, 4, 1686, 10.21105/joss.01686 Pedersen Kuhn Wickham, 2016 Singh Zhang, 2017, Prediction of emergency department hospital admission based on natural language processing and neural networks∗, Method Inform Med, 56, 377, 10.3414/ME17-01-0024 Lee, 2020, Prediction of emergency department patient disposition decision for proactive resource allocation for admission, Health Care Manag Sci, 23, 339, 10.1007/s10729-019-09496-y Zwank, 2021, Provider-in-triage prediction of hospital admission after brief patient interaction, Am J Emerg Med, 40, 60, 10.1016/j.ajem.2020.11.072 Chang, 2018, Hospital strategies for reducing emergency department crowding: a mixed-methods study, Ann Emerg Med, 71, 497, 10.1016/j.annemergmed.2017.07.022 Baugh, 2021, The cases not seen: Patterns of emergency department visits and procedures in the era of COVID-19, Am J Emerg Med, 46, 476, 10.1016/j.ajem.2020.10.081 Wong, 2021, Quantification of sepsis model alerts in 24 US hospitals before and during the COVID-19 pandemic, JAMA Netw Open, 4, 10.1001/jamanetworkopen.2021.35286 Finlayson, 2021, The clinician and dataset shift in artificial intelligence, N Engl J Med, 385, 283, 10.1056/NEJMc2104626 Duckworth, 2021, Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19, Sci Rep-uk, 11 Kamran, 2022, Early identification of patients admitted to hospital for COVID-19 at risk of clinical deterioration: model development and multisite external validation study, BMJ, 376 Dayan, 2021, Federated learning for predicting clinical outcomes in patients with COVID-19, Nat Med, 27, 1735, 10.1038/s41591-021-01506-3