Learning classification models of cognitive conditions from subtle behaviors in the digital Clock Drawing Test
Tóm tắt
The Clock Drawing Test—a simple pencil and paper test—has been used for more than 50 years as a screening tool to differentiate normal individuals from those with cognitive impairment, and has proven useful in helping to diagnose cognitive dysfunction associated with neurological disorders such as Alzheimer’s disease, Parkinson’s disease, and other dementias and conditions. We have been administering the test using a digitizing ballpoint pen that reports its position with considerable spatial and temporal precision, making available far more detailed data about the subject’s performance. Using pen stroke data from these drawings categorized by our software, we designed and computed a large collection of features, then explored the tradeoffs in performance and interpretability in classifiers built using a number of different subsets of these features and a variety of different machine learning techniques. We used traditional machine learning methods to build prediction models that achieve high accuracy. We operationalized widely used manual scoring systems so that we could use them as benchmarks for our models. We worked with clinicians to define guidelines for model interpretability, and constructed sparse linear models and rule lists designed to be as easy to use as scoring systems currently used by clinicians, but more accurate. While our models will require additional testing for validation, they offer the possibility of substantial improvement in detecting cognitive impairment earlier than currently possible, a development with considerable potential impact in practice.
Tài liệu tham khảo
ABS Consulting. (2002). Marine safety: Tools for risk-based decision making. New York: Rowman & Littlefield.
Albert, M. S., DeKosky, S. T., Dickson, D., Dubois, B., Feldman, H. H., Fox, N. C., et al. (2011). The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement, 7(3), 270–279.
Andrade, J. T. (2009). Handbook of violence risk assessment and treatment: New approaches for mental health professionals. Berlin: Springer.
Battistin, L., & Cagnin, A. (2010). Vascular cognitive disorder. A biological and clinical overview. Neurochemical Research, 35(12), 1933–1938.
Borgelt, C. (2005). An implementation of the FP-growth algorithm. In Proceedings of the 1st international workshop on open source data mining: Frequent pattern mining implementations, OSDM ’05 (pp. 1–5).
Borson, S., Scanlan, J., Brush, M., Vitaliano, P., & Dokmak, A. (2000). The mini-cog: a cognitive ‘vital signs’ measure for dementia screening in multi-lingual elderly. International Journal of Geriatric Psychiatry, 15(11), 1021–1027.
Breiman, L. (2001). Random forests. Machine learning, 45(1), 5–32.
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Belmont: Wadsworth.
Cohen, J., Penney, D. L., Davis, R., Libon, D. J., Swenson, R. A., Ajilore, O., et al. (2014). Digital clock drawing: Differentiating ‘thinking’ versus ‘doing’ in younger and older adults with depression. Journal of the International Neuropsychological Society, 20(09), 920–928.
Davis, R., & Penney, D.L. (2014). Method and apparatus for measuring representational motions in a medical context. US Patent 8,740,819.
Davis, R., Penney, D., Pittman, D., Libon, D., Swenson, R., & Kaplan, E. (2011). The Digital Clock Drawing Test (dCDT)—I: Development of a new computerized quantitative system. Presented at the 39th annual meeting of the International Neuropsychological Society, Boston, MA.
Davis, R., Libon, D. J., Au, R., Pitman, D., & Penney, D. L. (2014). THink: Inferring cognitive status from subtle behaviors. In Twenty-sixth IAAI conference (pp. 2898–2905).
Fan, R. E., Chang, K. W., Hsieh, C. J., Wang, X. R., & Lin, C. J. (2008). LIBLINEAR: A library for large linear classification. The Journal of Machine Learning Research, 9, 1871–1874.
Freedman, M., Leach, L., Kaplan, E., Winocur, G., Shulman, K. I., & Delis, D. C. (1994). Clock drawing: A neuropsychological analysis. Oxford: Oxford University Press.
Freitas, A. A., Wieser, D. C., & Apweiler, R. (2010). On the importance of comprehensible classification models for protein function prediction. IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB), 7(1), 172–182.
Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 1189–1232.
Grande, L., Rudolph, J., Davis, R., Penney, D., Price, C., & Swenson, R. (2013). Clock drawing: Standing the test of time. In Ashendorf Le Swenson (Ed.), The Boston process approach to neuropsychological assessment. Oxford: Oxford University Press.
Haury, A. C., Gestraud, P., & Vert, J. P. (2011). The influence of feature selection methods on accuracy, stability and interpretability of molecular signatures. PloS ONE, 6(12), e28,210.
Hauser, J. R., Toubia, O., Evgeniou, T., Befurt, R., & Dzyabura, D. (2010). Disjunctions of conjunctions, cognitive simplicity, and consideration sets. Journal of Marketing Research, 47(3), 485–496.
Joachims, T. (1998). Making large-scale SVM learning practical. LS8-report 24, Universität Dortmund, LS VIII-report.
Kim, H., Cho, Y. S., & Do, E. Y. L. (2011a). Computational clock drawing analysis for cognitive impairment screening. In Proceedings of the fifth international conference on tangible, embedded, and embodied interaction. ACM (pp. 297–300).
Kim, H., Cho, Y. S., & Do, E. Y. L. (2011b). Using pen-based computing in technology for health. Human–computer interaction. Users and applications (pp. 192–201). Berlin: Springer.
Lamar, M., Grajewski, M., Penney, D., Davis, R., Libon, D., & Kumar, A. (2011). The impact of vascular risk and depression on executive planning and production during graphomotor output across the lifespan. Paper at 5th Congress of the International Society for Vascular, Cognitive and Behavioural Disorders, Lille, France.
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015). Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model. Annals of Applied Statistics (accepted). http://imstat.org/aoas/next_issue.html.
Libon, D. J., Swenson, R. A., Barnoski, E. J., & Sands, L. P. (1993). Clock drawing as an assessment tool for dementia. Archives of Clinical Neuropsychology, 8(5), 405–415.
Lourenço, R. A., Ribeiro-Filho, S. T., Moreira, Id F H, Paradela, E. M. P., & Miranda, A Sd. (2008). The Clock Drawing Test: Performance among elderly with low educational level. Revista Brasileira de Psiquiatria, 30(4), 309–315.
Manos, P. J., & Wu, R. (1994). The ten point clock test: A quick screen and grading method for cognitive impairment in medical and surgical patients. The International Journal of Psychiatry in Medicine, 24(3), 229–244.
Markatou, M., Tian, H., Biswas, S., & Hripcsak, G. (2005). Analysis of variance of cross-validation estimators of the generalization error. The Journal of Machine Learning Research, 6, 1127–1168.
Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. (2007). Comprehensible credit scoring models using rule extraction from support vector machines. European Journal of Operational Research, 183(3), 1466–1476.
Mendez, M. F., Ala, T., & Underwood, K. L. (1992). Development of scoring criteria for the clock drawing task in Alzheimer’s disease. Journal of the American Geriatrics Society, 40(11), 1095–1099.
Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., et al. (2005). The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53(4), 695–699.
Nyborn, J. A., Himali, J. J., Beiser, A. S., Devine, S. A., Du, Y., Kaplan, E., et al. (2013). The Framingham Heart Study clock drawing performance: Normative data from the offspring cohort. Experimental Aging Research, 39(1), 80–108.
Peng, H., Long, F., & Ding, C. (2005). Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1226–1238.
Penney, D., Libon, D., Lamar, M., Price, C., Swenson, R., Eppig, J., et al. (2011a). The Digital Clock Drawing Test (dCDT)—I: Information contained within the “noise”, 5th Congress of the 1International Society for Vascular, Cognitive and Behavioural Disorders (VAS-COG), Lille, France.
Penney, D., Libon, D., Lamar, M., Price, C., Swenson, R., Scala, S., et al. (2011b). The Digital Clock Drawing Test (dCDT)—IV: Clock drawing time and hand placement latencies in mild cognitive impairment and dementia, abstract and poster. In 5th Congress of the International Society for Vascular, Cognitive and Behavioural Disorders, Lille, France.
Penney, D., Lamar, M., Libon, D., Price, C., Swenson, R., Scala, S., et al. (2013). The Digital Clock Drawing Test (dCDT)—Hooklets: A novel graphomotor measure of executive function, abstract and poster. In 6th Congress of the International Society for Vascular, Cognitive and Behavioural Disorders, Montreal, Canada.
Petersen, R., Caracciolo, B., Brayne, C., Gauthier, S., Jelic, V., & Fratiglioni, L. (2014). Mild cognitive impairment: A concept in evolution. Journal of Internal Medicine, 275(3), 214–228.
Plassman, B. L., Langa, K. M., Fisher, G. G., Heeringa, S. G., Weir, D. R., Ofstedal, M. B., et al. (2007). Prevalence of dementia in the United States: The aging, demographics, and memory study. Neuroepidemiology, 29(1–2), 125–132.
Price, C. C., Cunningham, H., Coronado, N., Freedland, A., Cosentino, S., Penney, D. L., et al. (2011). Clock drawing in the Montreal Cognitive Assessment: Recommendations for dementia assessment. Dementia and Geriatric Cognitive Disorders, 31(3), 179–187.
Prince, M., Guerchet, M., & Prina, M. (2013). Policy brief for heads of government: The global impact of dementia 2013–2050. London: Alzheimer Disease International.
Quinlan, J. R. (1993). C4.5: Programs for machine learning. Los Altos: Morgan Kaufmann.
Ridgeway, G. (2013). The pitfalls of prediction. NIJ Journal, National Institute of Justice, 271, 34–40.
Rouleau, I., Salmon, D. P., Butters, N., Kennedy, C., & McGuire, K. (1992). Quantitative and qualitative analyses of clock drawings in Alzheimer’s and Huntington’s disease. Brain and Cognition, 18(1), 70–87.
Royall, D. R., Cordes, J. A., & Polk, M. (1998). CLOX: An executive clock drawing task. Journal of Neurology, Neurosurgery & Psychiatry, 64(5), 588–594.
Shulman, K. I., Pushkar Gold, D., Cohen, C. A., & Zucchero, C. A. (1993). Clock-drawing and dementia in the community: A longitudinal study. International Journal of Geriatric Psychiatry, 8(6), 487–496.
Steinhart, D. (2006). Juvenile detention risk assessment: A practice guide to juvenile detention reform. Juvenile Detention Alternatives Initiative A project of the Annie E Casey Foundation, 28, 2011.
Storey, J. E., Rowland, J. T., Basic, D., & Conforti, D. A. (2001). A comparison of five clock scoring methods using ROC (receiver operating characteristic) curve analysis. International Journal of Geriatric Psychiatry, 16(4), 394–399.
Storey, J. E., Rowland, J. T., Basic, D., & Conforti, D. A. (2002). Accuracy of the clock drawing test for detecting dementia in a multicultural sample of elderly Australian patients. International Psychogeriatrics, 14(03), 259–271.
Strub, R. L., Black, F. W., & Strub, A. C. (1985). The mental status examination in neurology. Philadelphia: FA Davis.
Sun, H. (2006). An accurate and interpretable bayesian classification model for prediction of hERG liability. ChemMedChem, 1(3), 315–322.
Sunderland, T., Hill, J. L., Mellow, A. M., Lawlor, B. A., Gundersheimer, J., Newhouse, P., et al. (1989). Clock drawing in Alzheimer’s disease: A novel measure of dementia severity. Journal of the American Geriatrics Society, 37(8), 725–729.
Tian, L., & Tibshirani, R. (2011). Adaptive index models for marker-based risk stratification. Biostatistics, 12(1), 68–86.
Tuokko, H., Hadjistavropoulos, T., Rae, S., & O’Rourke, N. (2000). A comparison of alternative approaches to the scoring of clock drawing. Archives of Clinical Neuropsychology, 15(2), 137–148.
Ustun, B., & Rudin, C. (2015). Supersparse linear integer models for optimized medical scoring systems. Machine Learning. doi:10.1007/s10994-015-5528-6.
Ustun, B., Tracà, S, & Rudin, C. (2013). Supersparse linear integer models for predictive scoring systems. In Proceedings of AAAI late breaking track
Van Belle, V. M., Van Calster, B., Timmerman, D., Bourne, T., Bottomley, C., Valentin, L., et al. (2012). A mathematical model for interpretable clinical decision support with applications in gynecology. PloS ONE, 7(3), e34,312.
Verbeke, W., Martens, D., Mues, C., & Baesens, B. (2011). Building comprehensible customer churn prediction models with advanced rule induction techniques. Expert Systems with Applications, 38(3), 2354–2364.
Wang, F., & Rudin, C. (2015). Falling rule lists. In Proceedings of artificial intelligence and statistics (AISTATS) (pp. 1013–1022).
Wang, T., Rudin, C., Doshi, F., Liu, Y., Klampfl, E., & MacNeille, P. (2015). Bayesian Or’s of And’s for interpretable classification with application to context aware recommender systems (submitted).