A new adaptive VR-based exergame for hand rehabilitation after stroke
Tóm tắt
The aim of this work is to present an adaptive serious game based on virtual reality (VR) for functional rehabilitation of the hand after stroke. The game focuses on simulating the palmar grasping exercise commonly used in clinical settings. The system’s design follows a user-centered approach, involving close collaboration with functional rehabilitation specialists and stroke patients. It uses the Leap motion controller to enable patient interaction in the virtual environment, which was created using the Unity 3D game engine. The system relies on hand gestures involving opening and closing movements to interact with virtual objects. It incorporates parameters to objectively measure participants’ performance throughout the game session. These metrics are used to personalize the game’s difficulty to each patient’s motor skills. To do this, we implemented an approach that dynamically adjusts the difficulty of the exergame according to the patient’s performance during the game session. To achieve this, we used an unsupervised machine learning technique known as clustering, in particular using the K-means algorithm. By applying this technique, we were able to classify patients’ performance into distinct groups, enabling us to assess their skill level and adapt the difficulty of the game accordingly. To evaluate the system’s effectiveness and reliability, we conducted a subjective evaluation involving 11 stroke patients. The standardized System Usability Scale (SUS) questionnaire was used to assess the system’s ease of use, while the Intrinsic Motivation Inventory (IMI) was used to evaluate the participants’ subjective experience with the system. Evaluations showed that our proposed system is usable and acceptable on a C-level scale, with a good adjective score, and the patients perceived a high intrinsic motivation.
Tài liệu tham khảo
Montoya, M.F., Munoz, J.E., Henao, O.A.: Enhancing virtual rehabilitation in upper limbs with biocybernetic adaptation: the effects of virtual reality on perceived muscle fatigue, game performance and user experience. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 740–747 (2020). https://doi.org/10.1109/TNSRE.2020.2968869
Then, J.W., Shivdas, S., Tunku Ahmad Yahaya, T.S., Ab Razak, N.I., Choo, P.T.: Gamification in rehabilitation of metacarpal fracture using cost-effective end-user device: a randomized controlled trial. J. Hand Therap. 33, 235–242 (2020). https://doi.org/10.1016/j.jht.2020.03.029
Goršič, M., Cikajlo, I., Novak, D.: Competitive and cooperative arm rehabilitation games played by a patient and unimpaired person: effects on motivation and exercise intensity. J NeuroEng. Rehabil. 14, 23 (2017). https://doi.org/10.1186/s12984-017-0231-4
Alarcón-Aldana, A.C., Callejas-Cuervo, M., Bo, A.P.L.: Upper limb physical rehabilitation using serious videogames and motion capture systems: a systematic review. Sensors. 20, 5989 (2020). https://doi.org/10.3390/s20215989
Bortone, I., Leonardis, D., Solazzi, M., Procopio, C., Crecchi, A., Bonfiglio, L., Frisoli, A.: Integration of serious games and wearable haptic interfaces for Neuro Rehabilitation of children with movement disorders: A feasibility study. In: 2017 International Conference on Rehabilitation Robotics (ICORR). pp. 1094–1099. IEEE, London (2017)
Avola, D., Cinque, L., Foresti, G.L., Marini, M.R.: An interactive and low-cost full body rehabilitation framework based on 3D immersive serious games. J. Biomed. Inform. 89, 81–100 (2019). https://doi.org/10.1016/j.jbi.2018.11.012
Ribé, J.T., Serrancolí, G., Arriaga, I.A., Alpiste, F., Iriarte, A., Margelí, C., Mayans, B., García, A., Morancho, S., Molas, C.: Design and Development of Biomedical Applications for Post-Stroke Rehabilitation Subjects
Ferreira, B., Menezes, P.: An Adaptive Virtual Reality-Based Serious Game for Therapeutic Rehabilitation. Int. J. Onl. Eng. 16, 63 (2020). https://doi.org/10.3991/ijoe.v16i04.11923
Pillai, M., Yang, Y., Ditmars, C., Subhash, H.M.: Artificial intelligence-based interactive virtual reality-assisted gaming system for hand rehabilitation. In: Deserno, T.M. and Chen, P.-H. (eds.) Medical Imaging 2020: Imaging Informatics for Healthcare, Research, and Applications. p. 18. SPIE, Houston, United States (2020)
Piette, P., Pasquier, J.: Réalité virtuelle et rééducation. Kinésithérapie, la Revue. 12, 38–41 (2012). https://doi.org/10.1016/j.kine.2012.07.003
Kim, W.-S., Cho, S., Ku, J., Kim, Y., Lee, K., Hwang, H.-J., Paik, N.-J.: Clinical Application of Virtual Reality for Upper Limb Motor Rehabilitation in Stroke: Review of Technologies and Clinical Evidence. JCM. 9, 3369 (2020). https://doi.org/10.3390/jcm9103369
Ning, H., Wang, Z., Li, R., Zhang, Y., Mao, L.: A Review on Serious Games for Exercise Rehabilitation. arXiv:2201.04984 [cs]. (2022)
Amorim, P., Sousa Santos, B., Dias, P., Silva, S., Martins, H.: Serious Games for Stroke Telerehabilitation of Upper Limb - A Review for Future Research. Int J Telerehab. 12, 65–76 (2020). https://doi.org/10.5195/ijt.2020.6326
Lawrence, E.S., Coshall, C., Dundas, R., Stewart, J., Rudd, A.G., Howard, R., Wolfe, C.D.A.: Estimates of the Prevalence of Acute Stroke Impairments and Disability in a Multiethnic Population. Stroke 32, 1279–1284 (2001). https://doi.org/10.1161/01.STR.32.6.1279
Avola, D., Cinque, L., Pannone, D.: Design of a 3D Platform for Immersive Neurocognitive Rehabilitation. Information 11, 134 (2020). https://doi.org/10.3390/info11030134
Zahabi, M., Abdul Razak, A.M.: Adaptive virtual reality-based training: a systematic literature review and framework. Virtual Reality 24, 725–752 (2020). https://doi.org/10.1007/s10055-020-00434-w
Debnath, B., O’Brien, M., Yamaguchi, M., Behera, A.: A review of computer vision-based approaches for physical rehabilitation and assessment. Multimedia Syst. 28, 209–239 (2022). https://doi.org/10.1007/s00530-021-00815-4
Rincon, A.L., Yamasaki, H., Shimoda, S.: Design of a video game for rehabilitation using motion capture, EMG analysis and virtual reality. In: 2016 International Conference on Electronics. Communications and Computers (CONIELECOMP), pp. 198–204. IEEE, Cholula, Mexico (2016)
Trombetta, M., Bazzanello Henrique, P.P., Brum, M.R., Colussi, E.L., De Marchi, A.C.B., Rieder, R.: Motion Rehab AVE 3D: A VR-based exergame for post-stroke rehabilitation. Computer Methods and Programs in Biomedicine. 151, 15-20 (2017). https://doi.org/10.1016/j.cmpb.2017.08.008
Hashim, S.H.M., Ismail, M., Manaf, H., Hanapiah, F.A.: Development of Dual Cognitive Task Virtual Reality Game Addressing Stroke Rehabilitation. In: Proceedings of the 2019 3rd International Conference on Virtual and Augmented Reality Simulations. pp. 21-25. ACM, Perth WN Australia (2019
Ma, M., Zheng, H.: Virtual Reality and Serious Games in Healthcare. In: Brahnam, S. and Jain, L.C. (eds.) Advanced Computational Intelligence Paradigms in Healthcare 6. Virtual Reality in Psychotherapy, Rehabilitation, and Assessment. pp. 169-192. Springer Berlin Heidelberg, Berlin, Heidelberg (2011)
Gonçalves, A.R., Muñoz, J.E., Gouveia, É.R., Cameirão, M. da S., Bermúdez i Badia, S.: Effects of prolonged multidimensional fitness training with exergames on the physical exertion levels of older adults. Vis Comput. 37, 19-30 (2021). https://doi.org/10.1007/s00371-019-01736-0
Hossain, M.S., Hardy, S., Alamri, A., Alelaiwi, A., Hardy, V., Wilhelm, C.: AR-based serious game framework for post-stroke rehabilitation. Multimedia Syst. 22, 659–674 (2016). https://doi.org/10.1007/s00530-015-0481-6
Streicher, A., Smeddinck, J.D.: Personalized and Adaptive Serious Games. In: Dörner, R., Göbel, S., Kickmeier-Rust, M., Masuch, M., Zweig, K. (eds.) Entertainment Computing and Serious Games, pp. 332–377. Springer International Publishing, Cham (2016)
Hocine, N., Gouaïch, A., Cerri, S.A., Mottet, D., Froger, J., Laffont, I.: Adaptation in serious games for upper-limb rehabilitation: an approach to improve training outcomes. User Model. User-Adap. Inter. 25, 65–98 (2015). https://doi.org/10.1007/s11257-015-9154-6
Thielbar, K.O., Lord, T.J., Fischer, H.C., Lazzaro, E.C., Barth, K.C., Stoykov, M.E., Triandafilou, K.M., Kamper, D.G.: Training finger individuation with a mechatronic-virtual reality system leads to improved fine motor control post-stroke. J NeuroEngineering Rehabil. 11, 171 (2014). https://doi.org/10.1186/1743-0003-11-171
Sinaga, K.P., Yang, M.-S.: Unsupervised K-Means Clustering Algorithm. IEEE. Access 8, 80716–80727 (2020). https://doi.org/10.1109/ACCESS.2020.2988796
Colombo, R., Pisano, F., Micera, S., Mazzone, A., Delconte, C., Carrozza, M.C., Dario, P., Minuco, G.: Robotic techniques for upper limb evaluation and rehabilitation of stroke patients. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 311–324 (2005). https://doi.org/10.1109/TNSRE.2005.848352
Sunderland, A., Tinson, D., Bradley, L., Hewer, R.L.: Arm function after stroke. An evaluation of grip strength as a measure of recovery and a prognostic indicator. Journal of Neurology, Neurosurgery & Psychiatry. 52, 1267-1272 (1989). https://doi.org/10.1136/jnnp.52.11.1267
Kim, H., Miller, L.M., Fedulow, I., Simkins, M., Abrams, G.M., Byl, N., Rosen, J.: Kinematic Data Analysis for Post-Stroke Patients Following Bilateral Versus Unilateral Rehabilitation With an Upper Limb Wearable Robotic System. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 153–164 (2013). https://doi.org/10.1109/TNSRE.2012.2207462
Kong, K.-H., Lee, J.: Temporal recovery and predictors of upper limb dexterity in the first year of stroke: A prospective study of patients admitted to a rehabilitation centre. NRE. 32, 345–350 (2013). https://doi.org/10.3233/NRE-130854
Bigoni, M., Baudo, S., Cimolin, V., Cau, N., Galli, M., Pianta, L., Tacchini, E., Capodaglio, P., Mauro, A.: Does kinematics add meaningful information to clinical assessment in post-stroke upper limb rehabilitation? A case report. J Phys Ther Sci. 28, 2408–2413 (2016). https://doi.org/10.1589/jpts.28.2408
Sucar, L.E., Luis, R., Leder, R., Hernáindez, J., Sáinchez, I.: Gesture therapy: A vision-based system for upper extremity stroke rehabilitation. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. pp. 3690-3693. IEEE, Buenos Aires (2010)
Housman, S.J., Scott, K.M., Reinkensmeyer, D.J.: A Randomized Controlled Trial of Gravity-Supported, Computer-Enhanced Arm Exercise for Individuals With Severe Hemiparesis. Neurorehabil. Neural Repair 23, 505–514 (2009). https://doi.org/10.1177/1545968308331148
Kong, K.-H., Loh, Y.-J., Thia, E., Chai, A., Ng, C.-Y., Soh, Y.-M., Toh, S., Tjan, S.-Y.: Efficacy of a Virtual Reality Commercial Gaming Device in Upper Limb Recovery after Stroke: A Randomized. Controlled Study. Topics in Stroke Rehabilitation. 23, 333–340 (2016). https://doi.org/10.1080/10749357.2016.1139796
Rand, D., Givon, N., Weingarden, H., Nota, A., Zeilig, G.: Eliciting Upper Extremity Purposeful Movements Using Video Games: A Comparison With Traditional Therapy for Stroke Rehabilitation. Neurorehabil. Neural Repair 28, 733–739 (2014). https://doi.org/10.1177/1545968314521008
Broeren, J., Claesson, L., Goude, D., Rydmark, M., Sunnerhagen, K.S.: Virtual Rehabilitation in an Activity Centre for Community-Dwelling Persons with Stroke. Cerebrovasc. Dis. 26, 289–296 (2008). https://doi.org/10.1159/000149576
Sin, H., Lee, G.: Additional Virtual Reality Training Using Xbox Kinect in Stroke Survivors with Hemiplegia. American Journal of Physical Medicine & Rehabilitation. 92, 871–880 (2013). https://doi.org/10.1097/PHM.0b013e3182a38e40
Kwon, J.-S., Park, M.-J., Yoon, I.-J., Park, S.-H.: Effects of virtual reality on upper extremity function and activities of daily living performance in acute stroke: A double-blind randomized clinical trial. NRE. 31, 379–385 (2012). https://doi.org/10.3233/NRE-2012-00807
Lee, S., Kim, Y., Lee, B.-H.: Effect of Virtual Reality-based Bilateral Upper Extremity Training on Upper Extremity Function after Stroke: A Randomized Controlled Clinical Trial: Bilateral Upper Extremity Training in Post Stroke. Occup. Ther. Int. 23, 357–368 (2016). https://doi.org/10.1002/oti.1437
Subramanian, S.K., Lourenço, C.B., Chilingaryan, G., Sveistrup, H., Levin, M.F.: Arm Motor Recovery Using a Virtual Reality Intervention in Chronic Stroke: Randomized Control Trial. Neurorehabil. Neural Repair 27, 13–23 (2013). https://doi.org/10.1177/1545968312449695
Zheng, C., Liao, W., Xia, W.: Effect of combined low-frequency repetitive transcranial magnetic stimulation and virtual reality training on upper limb function in subacute stroke: a double-blind randomized controlled trail. J. Huazhong Univ. Sci. Technol. [Med. Sci.]. 35, 248-254 (2015). https://doi.org/10.1007/s11596-015-1419-0
Piron, L., Turolla, A., Agostini, M., Zucconi, C., Cortese, F., Zampolini, M., Zannini, M., Dam, M., Ventura, L., Battauz, M., Tonin, P.: Exercises for paretic upper limb after stroke: A combined virtual-reality and telemedicine approach. J. Rehabil. Med. 41, 1016–102 (2009). https://doi.org/10.2340/16501977-0459
Brunner, I., Skouen, J.S., Hofstad, H., Aßmus, J., Becker, F., Sanders, A.-M., Pallesen, H., Qvist Kristensen, L., Michielsen, M., Thijs, L., Verheyden, G.: Virtual Reality Training for Upper Extremity in Subacute Stroke (VIRTUES): A multicenter RCT. Neurology 89, 2413–2421 (2017). https://doi.org/10.1212/WNL.0000000000004744
Choi, Y.-H., Ku, J., Lim, H., Kim, Y.H., Paik, N.-J.: Mobile game-based virtual reality rehabilitation program for upper limb dysfunction after ischemic stroke. RNN. 34, 455–463 (2016). https://doi.org/10.3233/RNN-150626
Kiper, P., Agostini, M., Luque-Moreno, C., Tonin, P., Turolla, A.: Reinforced Feedback in Virtual Environment for Rehabilitation of Upper Extremity Dysfunction after Stroke: Preliminary Data from a Randomized Controlled Trial. Biomed. Res. Int. 2014, 1–8 (2014). https://doi.org/10.1155/2014/752128
Ma, M., Bechkoum, K.: Serious games for movement therapy after stroke. In: 2008 IEEE International Conference on Systems. Man and Cybernetics, pp. 1872–1877. IEEE, Singapore, Singapore (2008)
Nasri, N., Orts-Escolano, S., Cazorla, M.: An sEMG-Controlled 3D Game for Rehabilitation Therapies: Real-Time Time Hand Gesture Recognition Using Deep Learning Techniques. Sensors. 20, 6451 (2020). https://doi.org/10.3390/s20226451
Dhiman, A., Solanki, D., Bhasin, A., Das, A., Lahiri, U.: An intelligent, adaptive, performance-sensitive, and virtual reality-based gaming platform for the upper limb. Comput Anim Virtual Worlds. 29, e1800 (2018). https://doi.org/10.1002/cav.1800
Vallejo, D., Gmez-Portes, C., Albusac, J., Glez-Morcillo, C., Castro-Schez, J.J.: Personalized Exergames Language: A Novel Approach to the Automatic Generation of Personalized Exergames for Stroke Patients. Appl. Sci. 10, 7378 (2020). https://doi.org/10.3390/app10207378
Afyouni, I., Murad, A., Einea, A.: Adaptive Rehabilitation Bots in Serious Games. Sensors. 20, 7037 (2020). https://doi.org/10.3390/s20247037
Aguilar-Lazcano, C.A., Rechy-Ramirez, E.J.: Performance analysis of Leap motion controller for finger rehabilitation using serious games in two lighting environments. Measurement 157, 107677 (2020). https://doi.org/10.1016/j.measurement.2020.107677
Rechy-Ramirez, E.J., Marin-Hernandez, A., Rios-Figueroa, H.V.: A human-computer interface for wrist rehabilitation: a pilot study using commercial sensors to detect wrist movements. Vis Comput. 35, 41–55 (2019). https://doi.org/10.1007/s00371-017-1446-x
Burke, J.W., McNeill, M.D.J., Charles, D.K., Morrow, P.J., Crosbie, J.H., McDonough, S.M.: Optimising engagement for stroke rehabilitation using serious games. Vis Comput. 25, 1085–1099 (2009). https://doi.org/10.1007/s00371-009-0387-4
de los Reyes-Guzmàin, A., Lozano-Berrio, V., Alvarez-Rodríguez, M., López-Dolado, E., Ceruelo-Abajo, S., Talavera-Díaz, F., Gil-Agudo, A.: RehabHand: Oriented-tasks serious games for upper limb rehabilitation by using Leap Motion Controller and target population in spinal cord injury. NeuroRehabilitation 48(3), 365–373 (2021). https://doi.org/10.3233/NRE-201598
Harvey, C., Selmanović, E., O’Connor, J., Chahin, M.: A comparison between expert and beginner learning for motor skill development in a virtual reality serious game. Vis Comput. 37, 3–17 (2021). https://doi.org/10.1007/s00371-019-01702-w
David, L., Bouyer, G., Otmane, S.: Towards a low-cost interactive system for motor self-rehabilitation after stroke. IJVR. 17, 40-45 (2017). https://doi.org/10.20870/IJVR.2017.17.2.2890
Li, C., Cheng, L., Yang, H., Zou, Y., Huang, F.: An Automatic Rehabilitation Assessment System for Hand Function Based on Leap Motion and Ensemble Learning. Cybern. Syst. 52, 3–25 (2021). https://doi.org/10.1080/01969722.2020.1827798
de Souza, M.R.S.B., Gonçalves, R.S., Carbone, G.: Feasibility and Performance Validation of a Leap Motion Controller for Upper Limb Rehabilitation. Robotics 10, 130 (2021). https://doi.org/10.3390/robotics10040130
Koter, K., Samowicz, M., Redlicka, J., Zubrycki, I.: Hand Measurement System Based on Haptic and Vision Devices towards Post-Stroke Patients. 23 (2022)
David, L.: Conception et évaluation d’un système de réalité virtuelle pour l’assistance à l’auto-rééducation motrice du membre supérieur post-AVC. 165
Orhan, U., Hekim, M., Ozer, M.: EEG signals classification using the K-means clustering and a multilayer perceptron neural network model. Expert Syst. Appl. 38, 13475–13481 (2011). https://doi.org/10.1016/j.eswa.2011.04.149
Bangor, A.: Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. 4, 10 (2009)
Franck, J.A., Timmermans, A.A.A., Seelen, H.A.M.: Effects of a dynamic hand orthosis for functional use of the impaired upper limb in sub-acute stroke patients: A multiple single case experimental design study. TAD. 25, 177–187 (2013). https://doi.org/10.3233/TAD-130374