Design, development, and evaluation of an interactive personalized social robot to monitor and coach post-stroke rehabilitation exercises

User Modeling and User-Adapted Interaction - Tập 33 - Trang 545-569 - 2023
Min Hun Lee1, Daniel P. Siewiorek2, Asim Smailagic2, Alexandre Bernardino3, Sergi Bermúdez i Badia4
1Singapore Management University, Singapore, Singapore
2Carnegie Mellon University, Pittsburgh, USA
3Instituto Superior Técnico, Lisbon, Portugal
4NOVA-LINCS, University of Madeira, Funchal, Portugal

Tóm tắt

Socially assistive robots are increasingly being explored to improve the engagement of older adults and people with disability in health and well-being-related exercises. However, even if people have various physical conditions, most prior work on social robot exercise coaching systems has utilized generic, predefined feedback. The deployment of these systems still remains a challenge. In this paper, we present our work of iteratively engaging therapists and post-stroke survivors to design, develop, and evaluate a social robot exercise coaching system for personalized rehabilitation. Through interviews with therapists, we designed how this system interacts with the user and then developed an interactive social robot exercise coaching system. This system integrates a neural network model with a rule-based model to automatically monitor and assess patients’ rehabilitation exercises and can be tuned with individual patient’s data to generate real-time, personalized corrective feedback for improvement. With the dataset of rehabilitation exercises from 15 post-stroke survivors, we demonstrated our system significantly improves its performance to assess patients’ exercises while tuning with held-out patient’s data. In addition, our real-world evaluation study showed that our system can adapt to new participants and achieved 0.81 average performance to assess their exercises, which is comparable to the experts’ agreement level. We further discuss the potential benefits and limitations of our system in practice.

Tài liệu tham khảo

Azenkot, S., Feng, C., Cakmak, M.: Enabling building service robots to guide blind people a participatory design approach. In: 2016 11th ACM/IEEE International Conference on Human–Robot Interaction (HRI), pp. 3–10. IEEE (2016) Baltrušaitis, T., Ahuja, C., Morency, L.P.: Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 423–443 (2019) Beer, J.M., Smarr, C.A., Chen, T.L., Prakash, A., Mitzner, T.L., Kemp, C.C., Rogers, W.A.: The domesticated robot: design guidelines for assisting older adults to age in place. In: Proceedings of the Seventh Annual ACM/IEEE International Conference on Human–Robot Interaction, pp. 335–342 (2012) Cirstea, M., Levin, M.F.: Compensatory strategies for reaching in stroke. Brain 123(5), 940–953 (2000) Dall, T.M., Gallo, P.D., Chakrabarti, R., West, T., Semilla, A.P., Storm, M.V.: An aging population and growing disease burden will require alarge and specialized health care workforce by 2025. Health Aff. 32(11), 2013–2020 (2013) Developers, F.: ffmpeg tool. http://ffmpeg.org (2016) Dietterich, T.G.: Ensemble methods in machine learning. In: International Workshop on Multiple Classifier Systems, pp. 1–15. Springer (2000) Fasola, J., Matarić, M.J.: A socially assistive robot exercise coach for the elderly. J. Hum. Robot Interact. 2(2), 3–32 (2013) Feigin, V.L., Norrving, B., Mensah, G.A.: Global burden of stroke. Circ. Res. 120(3), 439–448 (2017) Feil-Seifer, D., Mataric, M.J.: Defining socially assistive robotics. In: 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005, pp. 465–468. IEEE (2005) Feingold Polak, R., Tzedek, S.L.: Social robot for rehabilitation: expert clinicians and post-stroke patients’ evaluation following a long-term intervention. In: Proceedings of the 2020 ACM/IEEE International Conference on Human–Robot Interaction, pp. 151–160 (2020) Gale, N.K., Heath, G., Cameron, E., Rashid, S., Redwood, S.: Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med. Res. Methodol. 13(1), 1–8 (2013) Gopinath, R.A.: Maximum likelihood modeling with gaussian distributions for classification. In: Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’98 (Cat. No. 98CH36181), vol. 2, pp. 661–664. IEEE (1998) Görer, B., Salah, A.A., Akın, H.L.: A robotic fitness coach for the elderly. In: International Joint Conference on Ambient Intelligence, pp. 124–139. Springer (2013) Görer, B., Salah, A.A., Akın, H.L.: An autonomous robotic exercise tutor for elderly people. Auton. Robot. 41(3), 657–678 (2017) Gouaillier, D., Hugel, V., Blazevic, P., Kilner, C., Monceaux, J., Lafourcade, P., Marnier, B., Serre, J., Maisonnier, B.: The nao humanoid: a combination of performance and affordability. CoRR arXiv:0807.3223 (2008) Guneysu, A., Arnrich, B.: Socially assistive child-robot interaction in physical exercise coaching. In: 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 670–675. IEEE (2017) Hasan, M., Roy-Chowdhury, A.K.: Continuous learning of human activity models using deep nets. In: European Conference on Computer Vision, pp. 705–720. Springer (2014) Irfan, B., Gomez, N.C., Casas, J., Senft, E., Gutiérrez, L.F., Rincon-Roncancio, M., Munera, M., Belpaeme, T., Cifuentes, C.A.: Using a personalised socially assistive robot for cardiac rehabilitation: a long-term case study. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 124–130. IEEE (2020) Kåringen, I., Dysvik, E., Furnes, B.: The elderly stroke patient’s long-term adherence to physiotherapy home exercises. Adv. Physiother. 13(4), 145–152 (2011) Kashi, S., Polak, R.F., Lerner, B., Rokach, L., Levy-Tzedek, S.: A machine-learning model for automatic detection of movement compensations in stroke patients. IEEE Trans. Emerg. Top. Comput. 9(3), 1234–1247 (2020) Këpuska, V., Bohouta, G.: Comparing speech recognition systems (Microsoft API, Google API and CMU Sphinx). Int. J. Eng. Res. Appl. 7(03), 20–24 (2017) Lee, M.H., Siewiorek, D.P., Smailagic, A., Bernardino, A., Bermúdez i Badia, S.: An exploratory study on techniques for quantitative assessment of stroke rehabilitation exercises. In: Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, UMAP ’20, pp. 303–307. ACM (2020) Lee, M.H., Siewiorek, D.P., Smailagic, A., Bernardino, A., Bermúdez i Badia, S.: Interactive hybrid approach to combine machine and human intelligence for personalized rehabilitation assessment. In: Proceedings of the ACM Conference on Health, Inference, and Learning, pp. 160–169 (2020) Lee, M.H., Siewiorek, D.P., Smailagic, A., Bernardino, A., Badia, S.B.: Towards personalized interaction and corrective feedback of a socially assistive robot for post-stroke rehabilitation therapy. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 1366–1373. IEEE (2020) Lee, M.H., Siewiorek, D.P., Smailagic, A., Bernardino, A., Badia, S.B.i.: Learning to assess the quality of stroke rehabilitation exercises. In: Proceedings of the 24th International Conference on Intelligent User Interfaces, pp. 218–228 (2019) Lee, M.H., Siewiorek, D.P., Smailagic, A., Bernardino, A., et al.: Designing personalized interaction of a socially assistive robot for stroke rehabilitation therapy. arXiv:2007.06473 (2020) Lee, M.H., Siewiorek, D.P., Smailagic, A., Bernardino, A., et al.: Enabling AI and robotic coaches for physical rehabilitation therapy: iterative design and evaluation with therapists and post-stroke survivors. Int. J. Soc. Robot. 1–22 (2022) Lin, J.F.S., Kulić, D.: Online segmentation of human motion for automated rehabilitation exercise analysis. IEEE Trans. Neural Syst. Rehabil. Eng. 22(1), 168–180 (2013) Matarić, M., Tapus, A., Winstein, C., Eriksson, J.: Socially assistive robotics for stroke and mild TBI rehabilitation. In: Advanced Technologies in Rehabilitation, pp. 249–262. IOS Press (2009) Matarić, M.J., Eriksson, J., Feil-Seifer, D.J., Winstein, C.J.: Socially assistive robotics for post-stroke rehabilitation. J. Neuroeng. Rehabil. 4(1), 5 (2007) Matarić, M.J., Scassellati, B.: Socially Assistive Robotics. Springer Handbook of Robotics, pp. 1973–1994 (2016) O’Sullivan, S.B., Schmitz, T.J., Fulk, G.: Physical Rehabilitation. F. A. Davis (2019) Pot, E., Monceaux, J., Gelin, R., Maisonnier, B.: Choregraphe: a graphical tool for humanoid robot programming. In: RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp. 46–51. IEEE (2009) Rao, R.B., Fung, G., Rosales, R.: On the dangers of cross-validation. an experimental evaluation. In: Proceedings of the 2008 SIAM International Conference on Data Mining, pp. 588–596. SIAM (2008) Riek, L.D.: Healthcare robotics. Commun. ACM 60(11), 68–78 (2017) Rudin, C., Radin, J.: Why are we using black box models in AI when we don’t need to? A lesson from an explainable AI competition (2019) Sanford, J., Moreland, J., Swanson, L.R., Stratford, P.W., Gowland, C.: Reliability of the Fugl–Meyer assessment for testing motor performance in patients following stroke. Phys. Ther. 73(7), 447–454 (1993) Schneider, S., Kummert, F.: Comparing robot and human guided personalization: adaptive exercise robots are perceived as more competent and trustworthy. Int. J. Soc. Robot. 13(2), 169–185 (2021) Tanguy, P., Rémy-Néris, O., et al.: Computational architecture of a robot coach for physical exercises in kinaesthetic rehabilitation. In: 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 1138–1143. IEEE (2016) Tapus, A., Maja, M., Scassellatti, B.: The grand challenges in socially assistive robotics (2007) Tapus, A., Mataric, M.J.: Towards socially assistive robotics. J. Robot. Soc. Jpn. 24(5), 576–578 (2006) Tapus, A., Tapus, C., Mataric, M.J.: The use of socially assistive robots in the design of intelligent cognitive therapies for people with dementia. In: 2009 IEEE International Conference on Rehabilitation Robotics, pp. 924–929. IEEE (2009) Weiss, K., Khoshgoftaar, T.M., Wang, D.: A survey of transfer learning. J. Big Data 3(1), 1–40 (2016) Winkle, K., Caleb-Solly, P., Turton, A., Bremner, P.: Social robots for engagement in rehabilitative therapies: Design implications from a study with therapists. In: Proceedings of the 2018 ACM/IEEE International Conference on Human–Robot Interaction, pp. 289–297. ACM (2018) Wolf, S.L., Catlin, P.A., Ellis, M., Archer, A.L., Morgan, B., Piacentino, A.: Assessing wolf motor function test as outcome measure for research in patients after stroke. Stroke 32(7), 1635–1639 (2001) Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., He, Q.: A comprehensive survey on transfer learning. Proc. IEEE 109(1), 43–76 (2020)