Integration of a Lightweight Customized 2D CNN Model to an Edge Computing System for Real-Time Multiple Gesture Recognition
Tóm tắt
The human-machine interface (HMI) collects electrophysiology signals incoming from the patient and utilizes them to operate the device. However, most applications are currently in the testing phase and are typically unavailable to everyone. Developing wearable HMI devices that are intelligent and more comfortable has been a focus of study in recent times. This work developed a portable, eight-channel electromyography (EMG) signal-based device that can distinguish 21 different types of motion. To identify the EMG signals, an analog front-end (AFE) integrated chip (IC) was created, and an integrated EMG signal acquisition device combining a stretchy wristband was made. Using the EMG movement signals of 10 volunteers, a SIAT database of 21 gestures was created. Using the SIAT dataset, a lightweight 2D CNN-LSTM model was developed and specialized training was given. The signal recognition accuracy is 96.4%, and the training process took a median of 14 min 13 s. The model may be used on lower-performance edge computing devices because of its compact size, and it is anticipated that it will eventually be applied to smartphone terminals.
Tài liệu tham khảo
Zheng, Y., Lv, X., Qian, L., Liu, X.: An optimal BP neural network track prediction method based on a GA–ACO hybrid algorithm. J. Mar. Sci. Eng. 10, 10 (2022)
Venugopalan, A., Reghunadhan, R.: Applying deep neural networks for the automatic recognition of sign language words: a communication aid to deaf agriculturists. Expert Syst. Appl. 185 (2021)
Alzubaidi, L., Zhang, J., Al-Dujaili, A., Humaidi, A.J., Duan, Y., Al-Shamma, O., Santamaría, L., Al-Amidie, M., Fadhel, M.A., Farhan, L.: Review of deep learning: concepts, cnn architectures, challenges, applications, future directions. J. Big Data. 8, 1 (2021)
Liu, Z., Wen, C., Su, Z., Liu, S., Sun, J., Kong, W.,…, Yang, Z.: Emotion-semantic-aware dual contrastive learning for epistemic emotion identification of learner-generated reviews in MOOCs. IEEE Trans. Neural Netw. Learn. Syst. (2023)
Qu, J., Mao, B., Li, Z., Xu, Y., Zhou, K., Cao, X., Wang, X.: Recent progress in advanced tactile sensing technologies for soft grippers. Adv. Funct. Mater. 33(41), 2306249 (2023)
Qu, J., Yuan, Q., Li, Z., Wang, Z., Xu, F., Fan, Q., Xu, M.: All-in-one strain-triboelectric sensors based on environment-friendly ionic hydrogel for wearable sensing and underwater soft robotic grasping. Nano Energy 111, 108387 (2023)
Fang, Y., Wang, K., Cheng, J., Lu, H.: A real-time hand gesture recognition method. In: 2007 IEEE International Conference on Multimedia and Expo 995–998 (2022)
Zeng, Q., Bie, B., Guo, Q., Yuan, Y., Han, Q., Han, X.,… Zhou, X.: Hyperpolarized Xe NMR signal advancement by metal-organic framework entrapment in aqueous solution. Proc. Natl. Acad. Sci. 117(30), 17558–17563 (2020)
Kattenborn, T., Leitloff, J., Schiefer, F., Hinz, S.: Review on convolutional neural networks (cnn) in vegetation remote sensing. ISPRS J. Photogramm. Remote Sens. 173, 24–49 (2021)
Bing, P., Liu, Y., Liu, W., Zhou, J., Zhu, L.: Electrocardiogram classification using TSST-based spectrogram and ConViT. Front. Cardiovasc. Med. 9, (2022)
Li, Y., Huang, J., Tian, F., Wang, H.-A., Dai, G.-Z.: Gesture interaction in virtual reality. Virtual Real. Intell. Hardw. 1, 84–112 (2019)
Li, J., Deng, Y., Sun, W., Li, W., Li, R., Li, Q.,… Liu, Z.: Resource orchestration of cloud-edge–based smart grid fault detection. ACM Trans. Sen. Netw. 18(3) (2022)
Mao, Y., You, C., Zhang, J., Huang, K., Letaief, K.B.: A survey on mobile edge computing: the communication perspective. IEEE Commun. Surv. Tutorials 19, 2322–2358 (2017)
Dai, X., Xiao, Z., Jiang, H., Alazab, M., Lui, J.C.S., Dustdar, S.,… Liu, J.: Task co-offloading for D2D-assisted mobile edge computing in industrial internet of things. IEEE Trans. Ind. Inform. 19(1), 480–490 (2023)
Jiang, H., Dai, X., Xiao, Z., Iyengar, A.K.: Joint task offloading and resource allocation for energy-constrained mobile edge computing. IEEE Trans. Mobile Comput. (2022)
Shen, X., Jiang, H., Liu, D., Yang, K., Deng, F., Lui, J.C.S.,… Luo, J.: PupilRec:Leveraging pupil morphology for recommending on smartphones. IEEE Internet Things J. 9(17), 15538–15553 (2022)
Jain, R., Karsh, R.K., Barbhuiya, A.A.: Literature review of vision8208;based dynamic gesture recognition using deep learning techniques. Concurr. Comput.: Pract. Exp. 34, 22 (2022)
Dai, X., Xiao, Z., Jiang, H., Lui, J.C.S.: UAV-assisted task offloading in vehicular edge computing networks. IEEE Trans. Mobile Comput. (2023)
Shi, W., Cao, J., Zhang, Q., Li, Y., Xu, L.: Edge computing: vision and challenges. IEEE Internet Things J. 3(5), 637–646 (2020)
Zhang, X., Huang, D., Li, H., Zhang, Y., Xia, Y.,… Liu, J.: Self-training maximum classifier discrepancy for EEG emotion recognition. CAAI Trans. Intell. Technol. (2023)
Li, B., Zhou, X., Ning, Z., Guan, X., Yiu, K.C.: Dynamic event-triggered security control for networked control systems with cyber-attacks: a model predictive control approach. Inf. Sci. 612, 384–398 (2022)
Yan, L., Shi, Y., Wei, M., Wu, Y.: Multi-feature fusing local directional ternary pattern for facial expressions signal recognition based on video communication system. Alexandria Eng. J. 63, 307–320 (2023)
Wang, F., Wang, H., Zhou, X., Fu, R.: A driving fatigue feature detection method based on multifractal theory. IEEE Sens. J. 22(19), 19046–19059 (2022)
Yu, J., Lu, L., Chen, Y., Zhu, Y., Kong, L.: An indirect eavesdropping Attack of keystrokes on touch screen through acoustic sensing. IEEE Trans. Mob. Comput. 20(2), 337–3512021 (2021)
Wang, Y., Han, X., Jin, S.: MAP based modeling method and performance study of a task offloading scheme with time-correlated traffic and VM repair in MEC systems. Wireless Netw. (2022)
Qu, Z., Liu, X., Zheng, M.: Temporal-spatial quantum graph convolutional neural network based on schrödinger approach for traffic congestion prediction. IEEE Trans. Intell. Transp. Syst. (2022)
Ni, Q., Guo, J., Wu, W., Wang, H., Wu, J.: Continuous influence-based community partition for social networks. IEEE Trans. Netw. Sci. Eng. 9(3), 1187–1197 (2022)
Zhang, J., Tang, Y., Wang, H., Xu, K.: ASRO-DIO: active subspace random optimization based depth inertial odometry. IEEE Trans. Robot. 1–13, (2022)
She, Q., Hu, R., Xu, J., Liu, M., Xu, K.,… Huang, H.: Learning High-DOF reaching-and-grasping via dynamic representation of gripper-object interaction. ACM Trans. Graph., 41(4), (2022)
Cheng, B., Wang, M., Zhao, S., Zhai, Z., Zhu, D.,… Chen, J.: Situation-aware dynamic service coordination in an IoT environment. IEEE/ACM Trans. Netw. 25(4), 2082–2095 (2017)
Zhuang, Y., Jiang, N., Xu, Y., Xiangjie, K., Kong, X.: Progressive distributed and parallel similarity retrieval of large ct image sequences in mobile telemedicine networks. Wirel. Commun. Mobile Comput. (2022)
Tang, Y., Liu, S., Deng, Y., Zhang, Y., Yin, L.,… Zheng, W.: An improved method for soft tissue modeling. Biomed. Signal Process. Control 65, (2021)
Lu, S., Yang, J., Bo, Y.: Analysis and design of surgical instrument localization algorithm. Comput. Model. Eng. Sci. 137(1), 669–685 (2023)
Lu, S., Liu, S., Hou, P., Yang, B., Liu, M., Yin, L.,… Zheng, W.: Soft tissue feature tracking based on deep matching network. Comput. Model. Eng. Sci. 136(1), 363–379 (2023)