Machine learning determination of applied behavioral analysis treatment plan type
Tóm tắt
Applied behavioral analysis (ABA) is regarded as the gold standard treatment for autism spectrum disorder (ASD) and has the potential to improve outcomes for patients with ASD. It can be delivered at different intensities, which are classified as comprehensive or focused treatment approaches. Comprehensive ABA targets multiple developmental domains and involves 20–40 h/week of treatment. Focused ABA targets individual behaviors and typically involves 10–20 h/week of treatment. Determining the appropriate treatment intensity involves patient assessment by trained therapists, however, the final determination is highly subjective and lacks a standardized approach. In our study, we examined the ability of a machine learning (ML) prediction model to classify which treatment intensity would be most suited individually for patients with ASD who are undergoing ABA treatment. Retrospective data from 359 patients diagnosed with ASD were analyzed and included in the training and testing of an ML model for predicting comprehensive or focused treatment for individuals undergoing ABA treatment. Data inputs included demographics, schooling, behavior, skills, and patient goals. A gradient-boosted tree ensemble method, XGBoost, was used to develop the prediction model, which was then compared against a standard of care comparator encompassing features specified by the Behavior Analyst Certification Board treatment guidelines. Prediction model performance was assessed via area under the receiver-operating characteristic curve (AUROC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The prediction model achieved excellent performance for classifying patients in the comprehensive versus focused treatment groups (AUROC: 0.895; 95% CI 0.811–0.962) and outperformed the standard of care comparator (AUROC 0.767; 95% CI 0.629–0.891). The prediction model also achieved sensitivity of 0.789, specificity of 0.808, PPV of 0.6, and NPV of 0.913. Out of 71 patients whose data were employed to test the prediction model, only 14 misclassifications occurred. A majority of misclassifications (n = 10) indicated comprehensive ABA treatment for patients that had focused ABA treatment as the ground truth, therefore still providing a therapeutic benefit. The three most important features contributing to the model’s predictions were bathing ability, age, and hours per week of past ABA treatment. This research demonstrates that the ML prediction model performs well to classify appropriate ABA treatment plan intensity using readily available patient data. This may aid with standardizing the process for determining appropriate ABA treatments, which can facilitate initiation of the most appropriate treatment intensity for patients with ASD and improve resource allocation.
Tài liệu tham khảo
APA—DSM—Diagnostic and Statistical Manual of Mental Disorders. (2013). https://www.appi.org/dsm
Hus Y, Segal O (2021) Challenges surrounding the diagnosis of autism in children. Neuropsychiatr Dis Treat 17:3509–3529. https://doi.org/10.2147/NDT.S282569
Lord C, Elsabbagh M, Baird G, Veenstra-Vanderweele J (2018) Autism spectrum disorder. Lancet 392(10146):508–520. https://doi.org/10.1016/S0140-6736(18)31129-2
Zeidan J, Fombonne E, Scorah J, Ibrahim A, Durkin MS, Saxena S, Yusuf A, Shih A, Elsabbagh M (2022) Global prevalence of autism: a systematic review update. Autism Res 15(5):778–790. https://doi.org/10.1002/aur.2696
Maenner, M. J., Shaw, K. A., Baio, J., EdS1, Washington, A., Patrick, M., DiRienzo, M., Christensen, D. L., Wiggins, L. D., Pettygrove, S., Andrews, J. G., Lopez, M., Hudson, A., Baroud, T., Schwenk, Y., White, T., Rosenberg, C. R., Lee, L.-C., Harrington, R. A., Dietz, P. M. (2020). Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years—Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2016. MMWR Surveillance Summaries. 69(4): 1–12. https://doi.org/10.15585/mmwr.ss6904a1
Kuo AA, Anderson KA, Crapnell T, Lau L, Shattuck PT (2018) Introduction to transitions in the life course of autism and other developmental disabilities. Pediatrics 141(Supplement_4):S267–S271. https://doi.org/10.1542/peds.2016-4300B
Shattuck P, Shea L (2022) Improving care and service delivery for autistic youths transitioning to adulthood. Pediatrics 149(Supplement 4):e2020049437I. https://doi.org/10.1542/peds.2020-049437I
Applied Behavior Analysis Treatment of Autism Spectrum Disorder: Practice Guidelines for Healthcare Funders and Managers from CASP | Cambridge Center for Behavioral Studies. (n.d.). Retrieved September 1, 2022, from https://behavior.org/casp-aba-treatment-guidelines/
Eldevik S, Hastings RP, Hughes JC, Jahr E, Eikeseth S, Cross S (2009) Meta-analysis of early intensive behavioral intervention for children with autism. J Clin Child Adolesc Psychol 38(3):439–450. https://doi.org/10.1080/15374410902851739
Linstead E, Dixon DR, Hong E, Burns CO, French R, Novack MN, Granpeesheh D (2017) An evaluation of the effects of intensity and duration on outcomes across treatment domains for children with autism spectrum disorder. Transl Psychiatry. https://doi.org/10.1038/tp.2017.207
Smith T, Iadarola S (2015) Evidence base update for autism spectrum disorder. J of Clin Child Adolesc Psychol 44(6):897–922. https://doi.org/10.1080/15374416.2015.1077448
Volkmar F, Siegel M, Woodbury-Smith M, King B, McCracken J, State M (2014) Practice parameter for the assessment and treatment of children and adolescents with autism spectrum disorder. J Am Acad Child Adolesc Psychiatry 53(2):237–257. https://doi.org/10.1016/j.jaac.2013.10.013
Chiri G, Warfield ME (2012) Unmet need and problems accessing core health care services for children with autism spectrum disorder. Matern Child Health J 16(5):1081–1091. https://doi.org/10.1007/s10995-011-0833-6
Conners B, Johnson A, Duarte J, Murriky R, Marks K (2019) Future directions of training and fieldwork in diversity issues in applied behavior analysis. Behav Anal Pract 12(4):767–776. https://doi.org/10.1007/s40617-019-00349-2
Deochand N, Fuqua RW (2016) BACB certification trends: state of the states (1999 to 2014). Behav Anal Pract 9(3):243–252. https://doi.org/10.1007/s40617-016-0118-z
Smith-Young J, Chafe R, Audas R (2020) “Managing the wait”: parents’ experiences in accessing diagnostic and treatment services for children and adolescents diagnosed with autism spectrum disorder. Health Services Insights 13:117863292090214. https://doi.org/10.1177/1178632920902141
Wise MD, Little AA, Holliman JB, Wise PH, Wang CJ (2010) Can state early intervention programs meet the increased demand of children suspected of having autism spectrum disorders? J Dev Behav Pediatr 31(6):469–476. https://doi.org/10.1097/DBP.0b013e3181e56db2
Zhang YX, Cummings JR (2020) Supply of certified applied behavior analysts in the United States: implications for service delivery for children with autism. Psychiatr Serv 71(4):385–388. https://doi.org/10.1176/appi.ps.201900058
Daou N (2014) Conducting behavioral research with children attending nonbehavioral intervention programs for autism: the case of Lebanon. Behav Anal Pract 7(2):78–90. https://doi.org/10.1007/s40617-014-0017-0
Keenan M, Dillenburger K, Röttgers HR, Dounavi K, Jónsdóttir SL, Moderato P, Schenk JJAM, Virués-Ortega J, Roll-Pettersson L, Martin N (2015) Autism and ABA: the Gulf between North America and Europe. Rev J Autism Dev Disord 2(2):167–183. https://doi.org/10.1007/s40489-014-0045-2
Liao Y, Dillenburger K, Hu X (2022) Behavior analytic interventions for children with autism: policy and practice in the United Kingdom and China. Autism 26(1):101–120. https://doi.org/10.1177/13623613211020976
Mazurek MO, Harkins C, Menezes M, Chan J, Parker RA, Kuhlthau K, Sohl K (2020) Primary care providers’ perceived barriers and needs for support in caring for children with autism. J Pediatr 221:240-245.e1. https://doi.org/10.1016/j.jpeds.2020.01.014
Hayes SA, Watson SL (2013) The impact of parenting stress: A meta-analysis of studies comparing the experience of parenting stress in parents of children with and without autism spectrum disorder. J Autism Dev Disord 43(3):629–642. https://doi.org/10.1007/s10803-012-1604-y
Yirmiya N, Shaked M (2005) Psychiatric disorders in parents of children with autism: a meta-analysis. J Child Psychol Psychiatr 46(1):69–83. https://doi.org/10.1111/j.1469-7610.2004.00334.x
Abbas H, Garberson F, Liu-Mayo S, Glover E, Wall DP (2020) Multi-modular AI approach to streamline autism diagnosis in young children. Sci Rep. https://doi.org/10.1038/s41598-020-61213-w
Cognoa—Leading the way for pediatric behavioral health. (n.d.). Cognoa. July 26, 2022, from https://cognoa.com/
Duda M, Daniels J, Wall DP (2016) Clinical evaluation of a novel and mobile autism risk assessment. J Autism Dev Disord 46(6):1953–1961. https://doi.org/10.1007/s10803-016-2718-4
Hyde KK, Novack MN, LaHaye N, Parlett-Pelleriti C, Anden R, Dixon DR, Linstead E (2019) Applications of supervised machine learning in autism spectrum disorder research: a review. Rev J Autism Devl Disord 6(2):128–146. https://doi.org/10.1007/s40489-019-00158-x
Kosmicki JA, Sochat V, Duda M, Wall DP (2015) Searching for a minimal set of behaviors for autism detection through feature selection-based machine learning. Transl Psychiatr. https://doi.org/10.1038/tp.2015.7
Megerian JT, Dey S, Melmed RD, Coury DL, Lerner M, Nicholls CJ, Sohl K, Rouhbakhsh R, Narasimhan A, Romain J, Golla S, Shareef S, Ostrovsky A, Shannon J, Kraft C, Liu-Mayo S, Abbas H, Gal-Szabo DE, Wall DP, Taraman S (2022) Evaluation of an artificial intelligence-based medical device for diagnosis of autism spectrum disorder. NPJ Digital Medicine 5:57. https://doi.org/10.1038/s41746-022-00598-6
Megerian, J. T., Dey, S., Melmed, R. D., Coury, D. L., Lerner, M., Nicholls, C., Sohl, K., Rouhbakhsh, R., Narasimhan, A., Romain, J., Golla, S., Shareef, S., Ostrovsky, A., Shannon, J., Kraft, C., Liu-Mayo, S., Abbas, H., Gal-Szabo, D. E., Wall, D. P., & Taraman, S. (2022b). Performance of Canvas Dx, a novel software-based autism spectrum disorder diagnosis aid for use in a primary care setting (P13–5.001). Neurology, 98(18 Supplement). https://n.neurology.org/content/98/18_Supplement/1025
Tariq Q, Daniels J, Schwartz JN, Washington P, Kalantarian H, Wall DP (2018) Mobile detection of autism through machine learning on home video: a development and prospective validation study. PLoS Med 15(11):e1002705. https://doi.org/10.1371/journal.pmed.1002705
Wall DP, Dally R, Luyster R, Jung J-Y, DeLuca TF (2012) Use of artificial intelligence to shorten the behavioral diagnosis of autism. PLoS ONE 7(8):e43855. https://doi.org/10.1371/journal.pone.0043855
Kohli M, Kar AK, Bangalore A, Ap P (2022) Machine learning-based ABA treatment recommendation and personalization for autism spectrum disorder: an exploratory study. Brain Inf 9(1):16. https://doi.org/10.1186/s40708-022-00164-6
Emmanuel T, Maupong T, Mpoeleng D, Semong T, Mphago B, Tabona O (2021) A survey on missing data in machine learning. J Big Data 8(1):140. https://doi.org/10.1186/s40537-021-00516-9
Keogh E, Mueen A (2017) Curse of dimensionality. In: Sammut C, Webb GI (eds) Encyclopedia of machine learning and data mining. Springer, US, Boston, pp 314–315. https://doi.org/10.1007/978-1-4899-7687-1_192
Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. 26.
Lundberg, S., & Lee, S.-I. (2017). A unified approach to interpreting model predictions (arXiv:1705.07874; Version 1). arXiv. https://doi.org/10.48550/arXiv.1705.07874
Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785–794. https://doi.org/10.1145/2939672.2939785
Barton C, Chettipally U, Zhou Y, Jiang Z, Lynn-Palevsky A, Le S, Calvert J, Das R (2019) Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs. Comput Biol Med 109:79–84. https://doi.org/10.1016/j.compbiomed.2019.04.027
Ghandian S, Thapa R, Garikipati A, Barnes G, Green-Saxena A, Calvert J, Mao Q, Das R (2022) Machine learning to predict progression of non-alcoholic fatty liver to non-alcoholic steatohepatitis or fibrosis. JGH Open 6(3):196–204. https://doi.org/10.1002/jgh3.12716
Kim S-H, Jeon E-T, Yu S, Oh K, Kim CK, Song T-J, Kim Y-J, Heo SH, Park K-Y, Kim J-M, Park J-H, Choi JC, Park M-S, Kim J-T, Choi K-H, Hwang YH, Kim BJ, Chung J-W, Bang OY, Jung J-M (2021) Interpretable machine learning for early neurological deterioration prediction in atrial fibrillation-related stroke. Sci Rep 11:20610. https://doi.org/10.1038/s41598-021-99920-7
Li K, Shi Q, Liu S, Xie Y, Liu J (2021) Predicting in-hospital mortality in ICU patients with sepsis using gradient boosting decision tree. Medicine 100(19):e25813. https://doi.org/10.1097/MD.0000000000025813
Mao Q, Jay M, Hoffman JL, Calvert J, Barton C, Shimabukuro D, Shieh L, Chettipally U, Fletcher G, Kerem Y, Zhou Y, Das R (2018) Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU. BMJ Open 8(1):e017833. https://doi.org/10.1136/bmjopen-2017-017833
Sahni G, Lalwani S (2021) Deep learning methods for the prediction of chronic diseases: a systematic review. In: Mathur R, Gupta CP, Katewa V, Jat DS, Yadav N (eds) Emerging trends in data driven computing and communications. Springer, Singapore, pp 99–110. https://doi.org/10.1007/978-981-16-3915-9_8
Thapa R, Iqbal Z, Garikipati A, Siefkas A, Hoffman J, Mao Q, Das R (2022) Early prediction of severe acute pancreatitis using machine learning. Pancreatology 22(1):43–50. https://doi.org/10.1016/j.pan.2021.10.003
Ergul Aydin, Z., & Kamisli Ozturk, Z. (2021). Performance analysis of XGBoost classifier with missing data
Shwartz-Ziv R, Armon A (2022) Tabular data: deep learning is not all you need. Information Fusion 81:84–90. https://doi.org/10.1016/j.inffus.2021.11.011
Refaeilzadeh, P., Tang, L., & Liu, H. (2009). Cross-Validation. In L. Liu & M. T. Özsu (Eds.), Encyclopedia of Database Systems (pp. 532–538). Springer US. https://doi.org/10.1007/978-0-387-39940-9_565
About us. (n.d.). Scikit-Learn. Retrieved September 6, 2022, from https://scikit-learn.org/stable/about.html
Friedman JH (2002) Stochastic gradient boosting. Comput Stat Data Anal 38(4):367–378. https://doi.org/10.1016/S0167-9473(01)00065-2
Bickel PJ, Li B, Tsybakov AB, van de Geer SA, Yu B, Valdés T, Rivero C, Fan J, van der Vaart A (2006) Regularization in statistics. TEST 15(2):271–344. https://doi.org/10.1007/BF02607055
Hazra A (2017) Using the confidence interval confidently. J Thoracic Dis 9(10):4125–4130. https://doi.org/10.21037/jtd.2017.09.14
Granpeesheh D, Dixon D, Tarbox J, Kaplan A, Wilke A (2009) The effects of age and treatment intensity on behavioral intervention outcomes for children with autism spectrum disorders. Res Autism Spectr Disord 3:1014–1022. https://doi.org/10.1016/j.rasd.2009.06.007
Granpeesheh D, Tarbox J, Dixon DR (2009) Applied behavior analytic interventions for children with autism: a description and review of treatment research. Ann Clin Psychiatry 21(3):162–173
Zwaigenbaum L, Bryson S, Lord C, Rogers S, Carter A, Carver L, Chawarska K, Constantino J, Dawson G, Dobkins K, Fein D, Iverson J, Klin A, Landa R, Messinger D, Ozonoff S, Sigman M, Stone W, Tager-Flusberg H, Yirmiya N (2009) Clinical assessment and management of toddlers with suspected autism spectrum disorder: insights from studies of high-risk infants. Pediatrics 123(5):1383–1391. https://doi.org/10.1542/peds.2008-1606
Bone D, Bishop S, Black MP, Goodwin MS, Lord C, Narayanan SS (2016) Use of machine learning to improve autism screening and diagnostic instruments: effectiveness, efficiency, and multi-instrument fusion. J Child Psychol Psychiatry 57(8):927–937. https://doi.org/10.1111/jcpp.12559
Raj S, Masood S (2020) Analysis and detection of autism spectrum disorder using machine learning techniques. Procedia Computer Science 167:994–1004. https://doi.org/10.1016/j.procs.2020.03.399
Nevada Department of Health and Human Services Division of Health Care Financing and Policy. (2015). Division of health care financing and policy applied behavior analysis summary. NV.gov. https://dhcfp.nv.gov/uploadedFiles/dhcfpnvgov/content/Pgms/CPT/ABAReports/ABA_Summary.pdf#:~:text=Focused%20ABA%20generally%20ranges%20from%2010-25%20hours%20per,30-40%20hours%20per%20week%20with%20a%20lower%20caseload