A multimodal framework based on deep belief network for human locomotion intent prediction

Jiayi Li1, Jianhua Zhang2, Kexiang Li3, Jian Cao1, Hui Li2
1School of Mechanical Engineering, Hebei University of Technology, Tianjin 300401, China
2School of Mechanical Engineering, University of Science and Technology Beijing, Beijing 100083, China
3School of Mechanical and Materials Engineering, North China University of Technology, Beijing 100144, China

Tóm tắt

Từ khóa


Tài liệu tham khảo

Sun W, Lin JW, Su SF, Wang N, Er MJ. Reduced adaptive fuzzy decoupling control for lower limb exoskeleton. IEEE Transact Cybernetics. 2021;51(3):1099–109.

Li Z, Zhao K, Zhang L, Wu X, Su CY. Human-in-the-loop control of a wearable lower limb exoskeleton for stable dynamic walking. IEEE/ASME Trans Mechatron. 2021;26(5):2700–11.

Ma Y, Wu XY, Yang SX, Dang C. Online gait planning of lower-limb exoskeleton robot for paraplegic rehabilitation considering weight transfer process. IEEE Trans Autom Sci Eng. 2021;18(2):414–25.

Anderson AJ, Hudak YF, Muir BC, Aubin PM. Design, control, and evaluation of a robotic ankle-foot prosthesis emulator. IEEE Transact Med Robot Bionics. 2023;5(3):741–52.

Zheng E, Wang Q, Qiao H. Locomotion mode recognition with robotic transtibial prosthesis in inter-session and inter-day applications. IEEE Transact Neural Syst Rehabilitation Eng. 2019;27(9):1836–45.

Li XL, Hao YF, Zhang JH, Wang C, Li D, Zhang JJ. Design, modeling and experiments of a variable stiffness soft robotic glove for stroke patients with clenched fist deformity. IEEE Robot Autom Lett. 2023;8(7):4044–51.

Gao Y, Jia B, Houston M, Zhang Y. Hybrid EEG-fNIRS brain computer interface based on common spatial pattern by using EEG-informed general linear model. IEEE Trans Instrum Meas. 2023;72:1–10.

Huang C, Xiao Y, Xu G. Predicting human intention-behavior through EEG signal analysis using multi-scale CNN. IEEE/ACM Trans Comput Biol Bioinf. 2021;18(5):1722–9.

Elsayed NE, Tolba AS, Rashad MZ, Belal T, Sarhan S. A deep learning approach for brain computer interaction-motor execution EEG signal classification. IEEE Access. 2021;9:101513–29.

Park S, Lee D, Chung WK, Kim K. Hierarchical motion segmentation through sEMG for continuous lower limb motions. IEEE Robot Autom Lett. 2019;4(4):4402–9.

Ryu J, Kim DH. Real-time gait subphase detection using an EMG signal graph matching (ESGM) algorithm based on EMG signals. Expert Syst Appl. 2017;85:357–65.

Gautam A, Panwar M, Biswas D, Acharyya A. MyoNet: a transfer-learning-based LRCN for lower limb movement recognition and knee joint angle prediction for remote monitoring of rehabilitation progress from sEMG. IEEE J Transl Eng Health Med. 2020;8:1–10.

Shi X, Qin P, Zhu J, Zhai M, Shi W. Feature extraction and classification of lower limb motion based on sEMG signals. IEEE Access. 2020;8:132882–92.

Kang I, Molinaro DD, Choi G, Camargo J, Young AJ. Subject-independent continuous locomotion mode classification for robotic hip exoskeleton applications. IEEE Trans Biomed Eng. 2022;69(10):3234–42.

Su BY, Wang J, Liu SQ, Sheng M, Xiang K. An improved motion intent recognition method for intelligent lower limb prosthesis driven by inertial motion capture data. Acta Autom Sin. 2018;46(7):1517–30.

Su BY, Wang J, Liu SQ. A CNN-based method for intent recognition using inertial measurement units and intelligent lower limb prosthesis. IEEE Trans Neural Syst Rehabil Eng. 2019;27(5):1032–42.

Narayan A, Reyes FA, Ren M, Haoyong Y. Real-time hierarchical classification of time series data for locomotion mode detection. IEEE J Biomed Health Inform. 2022;26(4):1749–60.

Papapicco V, Chen B. A classification approach based on directed acyclic graph to predict locomotion activities with one inertial sensor on the thigh. IEEE Transact Med Robot Bionics. 2021;3(2):436–45.

Attal F, Amirat Y, Chibani A, Mohammed S. Automatic recognition of gait phases using a multiple-regression hidden markov model. IEEE/ASME Trans Mechatron. 2018;23(4):1597–607.

Huang L, Zheng J, Hu H. A gait phase detection method in complex environment based on DTW-mean templates. IEEE Senors J. 2021;21(13):15114–23.

Hu H, Zheng JB, Zhan E, Yu L. Curve similarity model for real-time gait phase detection based on ground contact forces. Sensors. 2019;19:3235.

Cui C, Bian G, Hou Z, Zhao J. A multimodal framework based on integration of cortical and muscular activities for decoding human intentions about lower limb motions. IEEE Trans Biomed Circuits Syst. 2017;11(4):889–99.

Tucker MR. Control strategies for active lower extremity prosthetics and orthotics: a review. J Neuroeng Rehabil. 2015;12(1):1–29.

Godiyal AK, Mondal M, Joshi SD. Force myography based novel strategy for locomotion classification. IEEE Transact Human-Machine Syst. 2018;48(6):648–57.

Al-Quraishi MS, Elamvazuthi I, Tang TB. Multimodal fusion approach based on EEG and EMG signals for lower limb movement recognition. IEEE Senors Journal. 2021;21(24):27640–50.

Wang Y, Cheng X, Jabban L, Sui X. Motion intention prediction and joint trajectories generation toward lower limb prostheses using EMG and IMU signals. IEEE Senors J. 2022;22(11):10719–29.

Camargo J, Flanagan W, Csomay-Shanklin N, Kanwar B, Young A. A machine learning strategy for locomotion classification and parameter estimation using fusion of wearable sensors. IEEE Trans Biomed Eng. 2021;68(5):1569–78.

Cui C, Bian GB, Hou ZG, Zhao J, Su GD, Zhou H, Peng L, Wang WQ. Simultaneous recognition and assessment of post-stroke hemiparetic gait by fusing kinematic, kinetic, and electrophysiological data. IEEE Transact Neural Syst Rehabil Eng. 2018;26(4):856–64.

Zhao Y, Wang J, Zhang Y, Liu H. Flexible and wearable EMG and PSD sensors enabled locomotion mode recognition for IoHT-based in-home rehabilitation. IEEE Senors J. 2021;21(23):26311–9.

Sun Y, Lo B. An artificial neural network framework for gait-based biometrics. IEEE J Biomed Health Inform. 2019;23(3):987–98.

Zhang K, Wang J, De Silva CW, Fu CL. Unsupervised cross-subject adaptation for predicting human locomotion intent. IEEE Transact Neural Syst Rehabil Eng. 2020;28(3):646–57.

Lee UH, Bi J, Patel R, Fouhey D. Image Transformation and cnns: a strategy for encoding human locomotor intent for autonomous wearable robots. IEEE Robot Automation Lett. 2020;5(4):5440–7.

Moon J, Jung J, Kang E, Choi SI. Open set user identification using gait pattern analysis based on ensemble deep neural network. IEEE Senors J. 2022;22(17):16975–84.

Hu B, Rouse E. Benchmark datasets for bilateral lower-limb neuromechanical signals from wearable sensors during unassisted locomotion in able-bodied individuals. Front Robot AI. 2018;5(14):1–5.

Pan T, Tsai WL, Chang CY, Yeh CW. A hierarchical hand gesture recognition framework for sports referee training-based EMG and accelerometer sensors. IEEE Transact Cybern. 2022;52(5):3172–83.

He H, Fan Z, Levi JH, Zhi D, Daniel RR, Kevin BE. Continuous Locomotion-Mode Identification for Prosthetic Legs Based on Neuromuscular-Mechanical Fusion. IEEE Transact Biomed Eng. 2011;58(10):2867–75.