Developing a machine learning model to identify delirium risk in geriatric internal medicine inpatients
Tóm tắt
To develop a machine learning model that predicts delirium risk in geriatric internal medicine inpatients. Depression, cognitive impairment, types of drugs, nutritional status, and activity of daily life (ADL) were important predictors of delirium. A machine learning model constructed from the above predictors achieved satisfactory discriminative ability. This machine learning model may allow more precise targeting of delirium prevention. To develop a machine learning model that predicts delirium risk in geriatric internal medicine inpatients. A prospective cohort study of internal medicine wards in a tertiary care hospital in China. Blinded observers assessed delirium using the Confusion Assessment Method (CAM). The data set was randomly divided into a training set (70%) and a test set (30%). The model was trained on the training set using the decision tree and the five-fold cross-validation, and then the model performance was evaluated on the test set. Under-sampling was used to address the class imbalance. The discriminatory power of the model was measured by the area under the receiver operating characteristic curve (AUC) and F1 score. The data set comprised 740 patients from March 2016 to January 2017. The training set included 518 patients; the median (IQR) age was 84 (79–87) years; 364 (70.3%) were men; 71 (13.7%) with delirium. The test set included 222 patients; the median (IQR) age was 84.5 (79–87) years; 163 (73.4%) were men; 30 (13.5%) with delirium. In total, the data set included 740 hospital admissions with a median (IQR) age of 84 (79–87) years, 527 (71.2%) were men, and 101 (13.6%) with delirium. From 32 potential predictors, we included five variables in the predictive model: depression, cognitive impairment, types of drugs, nutritional status, and activity of daily life (ADL). The mean AUC on the training set was 0.967, the AUC and F1 score on the test set was 0.950 and 0.810, respectively. The model achieved 93.3% sensitivity, 94.3% specificity, 71.8% positive predictive value, 98.9% negative predictive value, and 94.1% accuracy on the test set. This machine learning model may allow more precise targeting of delirium prevention and could support clinical decision making in geriatric internal medicine wards.
Tài liệu tham khảo
Inouye SK, Westendorp RGJ, Saczynski JS (2014) Delirium in elderly people. Lancet 383:911–922. https://doi.org/10.1016/S0140-6736(13)60688-1
Anand A, MacLullich AMJ (2017) Delirium in hospitalized older adults Medicine. N Engl J Med 45:46–50. https://doi.org/10.1016/J.MPMED.2016.10.006
Ahmed S, Leurent B, Sampson EL (2014) Risk factors for incident delirium among older people in acute hospital medical units: a systematic review and meta-analysis. Age Ageing 43:326–333. https://doi.org/10.1093/AGEING/AFU022
Liao L, Mark DB (2003) Clinical prediction models: are we building better mousetraps? J Am Coll Cardiol 42:851–853. https://doi.org/10.1016/S0735-1097(03)00836-2
Inouye SK, Zhang Y, Jones RN, Kiely DK, Yang F, Marcantonio ER (2007) Risk factors for delirium at discharge: development and validation of a predictive model. Arch Intern Med 167:1406–1413. https://doi.org/10.1001/ARCHINTE.167.13.1406
Martinez JA, Belastegui A, Basabe I, Goicoechea X, Aguirre C, Lizeaga N et al (2012) Derivation and validation of a clinical prediction rule for delirium in patients admitted to a medical ward: an observational study. BMJ Open 2:e001599. https://doi.org/10.1136/BMJOPEN-2012-001599
Isfandiaty R, Harimurti K, Setiati S, Roosheroe AG (2012) Incidence and predictors for delirium in hospitalized elderly patients: a retrospective cohort study. Acta Medica Indonesiana 44:290–297
Douglas VC, Hessler CS, Dhaliwal G, Betjemann JP, Fukuda KA, Alameddine LR et al (2013) The AWOL tool: derivation and validation of a delirium prediction rule. J Hosp Med 8:493–499. https://doi.org/10.1002/JHM.2062
Carrasco MP, Villarroel L, Andrade M, Calderón J, González M (2014) Development and validation of a delirium predictive score in older people. Age Ageing 43:346–351. https://doi.org/10.1093/AGEING/AFT141
Hein C, Forgues A, Piau A, Sommet A, Nourhashémi F, Vellas B (2014) Impact of polypharmacy on occurrence of delirium in elderly emergency patients. J Am Med Dir Assoc 15:850.e11-850.e15. https://doi.org/10.1016/J.JAMDA.2014.08.012
Waljee AK, Higgins PDR (2010) Machine learning in medicine: a primer for physicians. Am J Gastroenterol 105:1224–1226. https://doi.org/10.1038/AJG.2010.173
Wong A, Young AT, Liang AS, Gonzales R, Douglas VC, Hadley D (2018) Development and validation of an electronic health record-based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment. JAMA Netw Open 1:e181018–e181018. https://doi.org/10.1001/JAMANETWORKOPEN.2018.1018
Corradi JP, Thompson S, Mather JF, Waszynski CM, Dicks RS (2018) Prediction of incident delirium using a random forest classifier. J Med Syst 42:1–10. https://doi.org/10.1007/S10916-018-1109-0
Oh J, Cho D, Park J, Na SH, Kim J, Heo J et al (2018) Prediction and early detection of delirium in the intensive care unit by using heart rate variability and machine learning. Physiol Meas 39:035004. https://doi.org/10.1088/1361-6579/AAAB07
Hercus C, Hudaib A-R (2020) Delirium misdiagnosis risk in psychiatry: a machine learning-logistic regression predictive algorithm. BMC Health Serv Res 20:1–7. https://doi.org/10.1186/S12913-020-5005-1
Davoudi A, Ozrazgat-Baslanti T, Ebadi A, Bursian AC, Bihorac A, Rashidi P. Delirium prediction using machine learning models on predictive electronic health records data. In: Proceedings—2017 IEEE 17th International Conference on Bioinformatics and Bioengineering, BIBE 2017 2017;2018-January:568–73. https://doi.org/10.1109/BIBE.2017.00014.
Racine AM, Tommet D, D’Aquila ML, Fong TG, Gou Y, Tabloski PA et al (2020) Machine learning to develop and internally validate a predictive model for post-operative delirium in a prospective, observational clinical cohort study of older surgical patients. J Gen Intern Med 36:265–73. https://doi.org/10.1007/S11606-020-06238-7
Nabeel H, Hirsch GM, Abidi SR, Abidi SSR. Exploiting machine learning algorithms and methods for the prediction of agitated delirium after cardiac surgery: models development and validation study. JMIR Med Inform 2019;7(4):E14993https://Medinform.Jmir.Org/2019/4/E14993 2019;7:e14993. https://doi.org/10.2196/14993.
Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI (1990) Clarifying confusion: the confusion assessment method: a new method for detection of delirium. Ann Intern Med 113:941–948. https://doi.org/10.7326/0003-4819-113-12-941
Rubenstein LZ, Harker JO, Salvà A, Guigoz Y, Vellas B (2001) Screening for undernutrition in geriatric practicedeveloping the short-form mini-nutritional assessment (MNA-SF). J Gerontol Ser A 56:M366–M372. https://doi.org/10.1093/GERONA/56.6.M366
Nyunt MSZ, Fones C, Niti M, Ng T-P (2009) Criterion-based validity and reliability of the Geriatric Depression Screening Scale (GDS-15) in a large validation sample of community-living Asian older adults. Aging Mental Health 13:376–82. https://doi.org/10.1080/13607860902861027
Pfeiffer E (1975) A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients†. J Am Geriatr Soc 23:433–441. https://doi.org/10.1111/J.1532-5415.1975.TB00927.X
F-P M, W-G D, S W (2011) Validation of the polish version of the Athens insomnia scale. Psychiatria Polska 45:211–21
Shah S, Vanclay F, Cooper B (1989) Improving the sensitivity of the Barthel Index for stroke rehabilitation. J Clin Epidemiol 42:703–709. https://doi.org/10.1016/0895-4356(89)90065-6
Hicks CL, von Baeyer CL, Spafford PA, van Korlaar I, Goodenough B (2001) The Faces Pain Scale—revised: toward a common metric in pediatric pain measurement. Pain 93:173–183. https://doi.org/10.1016/S0304-3959(01)00314-1
McSherry D (1999) Strategic induction of decision trees. Knowl Based Syst 12:269–275. https://doi.org/10.1016/S0950-7051(99)00024-6
Quinlan J (1992) C4.5: programs for machine learning, Morgan Kaufmann, San Mateo, CA
Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and Regression Trees. Wadsworth International Group, Belmont, CA
Wasikowski M, Chen XW (2010) Combating the small sample class imbalance problem using feature selection. IEEE Trans Knowl Data Eng 22:1388–1400. https://doi.org/10.1109/TKDE.2009.187
Beap A, Prathi C, Carolina M (2004) A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explor Newslett 6:20–9. https://doi.org/10.1145/1007730.1007735
Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–57. https://doi.org/10.1613/JAIR.953
García S, Luengo J, Herrera F (2016) Tutorial on practical tips of the most influential data preprocessing algorithms in data mining. Knowl Based Syst 98:1–29. https://doi.org/10.1016/J.KNOSYS.2015.12.006
Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Inf Process Manage 45:427–437. https://doi.org/10.1016/J.IPM.2009.03.002
O’Sullivan R, Inouye SK, Meagher D (2014) Delirium and depression: inter-relationship and clinical overlap in elderly people. Lancet Psychiatry 1:303–311. https://doi.org/10.1016/S2215-0366(14)70281-0
Viktil KK, Blix HS, Moger TA, Reikvam A (2007) Polypharmacy as commonly defined is an indicator of limited value in the assessment of drug-related problems. Br J Clin Pharmacol 63:187–195. https://doi.org/10.1111/J.1365-2125.2006.02744.X
Vreeswijk R, Kalisvaart K, Kalisvaart I (2014) Development and validation of the Delirium Risk Assessment Score (DRAS). J Geriatr Oncol 5:S48–S49. https://doi.org/10.1016/J.JGO.2014.09.082
Carrière I, Fourrier-Reglat A, Dartigues J-F, Rouaud O, Pasquier F, Ritchie K et al (2009) Drugs with anticholinergic properties, cognitive decline, and dementia in an elderly general population: the 3-city study. Arch Intern Med 169:1317–1324. https://doi.org/10.1001/ARCHINTERNMED.2009.229
Glass J, Lanctôt KL, Herrmann N, Sproule BA, Busto UE (2005) Sedative hypnotics in older people with insomnia: meta-analysis of risks and benefits. BMJ 331:1169. https://doi.org/10.1136/BMJ.38623.768588.47
Stahlmann R, Lode H (2010) Safety considerations of fluoroquinolones in the elderly. Drugs Aging 27:193–209. https://doi.org/10.2165/11531490-000000000-00000
Oh ES, Fong TG, Hshieh TT, Inouye SK (2017) Delirium in older persons: advances in diagnosis and treatment. JAMA 318:1161–1174. https://doi.org/10.1001/JAMA.2017.12067
Peduzzi P, Concato J, Feinstein AR, Holford TR (1995) Importance of events per independent variable in proportional hazards regression analysis II. Accuracy and precision of regression estimates. J Clin Epidemiol 48:1503–10. https://doi.org/10.1016/0895-4356(95)00048-8
Peduzzi P, Concato J, Kemper E, Holford TR, Feinstem AR (1996) A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 49:1373–1379. https://doi.org/10.1016/S0895-4356(96)00236-3
Vittinghoff E, McCulloch CE (2007) Relaxing the rule of ten events per variable in logistic and cox regression. Am J Epidemiol 165:710–718. https://doi.org/10.1093/AJE/KWK052