Nâng cao nhận diện hoạt động của con người bằng cách sử dụng học sâu và dữ liệu chuỗi thời gian được tăng cường

Journal of Ambient Intelligence and Humanized Computing - Tập 12 - Trang 10565-10580 - 2021
Hongtao Lu1, Mohammad Al-Zinati2, Mahmoud Al-Ayyoub2, Luay Alawneh2, Tamam Alsarhan1,2, Yaser Jararweh2
1Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China
2Faculty of Computer and Information Technology, Jordan University of Science and Technology, Irbid, Jordan

Tóm tắt

Nhận diện hoạt động của con người liên quan đến việc phát hiện các loại chuyển động và hành động khác nhau của con người bằng cách sử dụng dữ liệu thu thập từ nhiều loại cảm biến khác nhau. Các phương pháp học sâu, khi được áp dụng trên dữ liệu chuỗi thời gian, cung cấp những kết quả hứa hẹn vượt trội so với các kỹ thuật trích xuất đặc trưng thủ công cần nhiều công sức, vốn phụ thuộc cao vào chất lượng của các tham số miền đã được xác định. Trong bài báo này, chúng tôi nghiên cứu lợi ích của việc tăng cường dữ liệu chuỗi thời gian trong việc cải thiện độ chính xác của một số mô hình học sâu trên dữ liệu hoạt động của con người thu thập từ cảm biến gia tốc của điện thoại di động. Cụ thể hơn, chúng tôi so sánh hiệu suất của các mô hình mạng nơ-ron Vanilla, Long-Short Term Memory (LSTM) và Gated Recurrent Units (GRU) trên ba bộ dữ liệu mã nguồn mở. Chúng tôi sử dụng hai kỹ thuật tăng cường dữ liệu chuỗi thời gian và nghiên cứu tác động của chúng đến độ chính xác của các mô hình mục tiêu. Các thí nghiệm cho thấy việc sử dụng Gated Recurrent Units đạt được kết quả tốt nhất về độ chính xác và thời gian huấn luyện, tiếp theo là kỹ thuật Long-Short Term Memory. Hơn nữa, các kết quả cho thấy việc sử dụng tăng cường dữ liệu cải thiện đáng kể chất lượng nhận diện.

Từ khóa

#Nhận diện hoạt động của con người #học sâu #dữ liệu chuỗi thời gian #tăng cường dữ liệu #mạng nơ-ron #cảm biến gia tốc

Tài liệu tham khảo

Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Kudlur M (2016) Tensorflow: A system for large-scale machine learning. In: 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16), pp 265–283 Alsarhan T, Alawneh L, Al-Zinati M, Al-Ayyoub M (2019) Bidirectional gated recurrent units for human activity recognition using accelerometer data. In: 2019 IEEE SENSORS, IEEE, pp 1–4 Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL (2013) A public domain dataset for human activity recognition using smartphones. In: Esann, p 3 Behera A, Hogg DC, Cohn AG (2012) Egocentric activity monitoring and recovery. Asian conference on computer vision. Springer, Berlin, pp 519–532 Bidargaddi N, Sarela A, Klingbeil L, Karunanithi M (2007) Detecting walking activity in cardiac rehabilitation by using accelerometer. In: 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information. IEEE, pp 555–560 Che Z, Purushotham S, Cho K, Sontag D, Liu Y (2018) Recurrent neural networks for multivariate time series with missing values. Sci Rep 8(1):1–12 Chen Y, Zhong K, Zhang J, Sun Q, Zhao X (2016) Lstm networks for mobile human activity recognition. 2016 International conference on artificial intelligence: technologies and applications. Atlantis Press, Paris Chen Z, Zhu Q, Soh YC, Zhang L (2017) Robust human activity recognition using smartphone sensors via CT-PCA and online SVM. IEEE Trans Industr Inf 13(6):3070–3080 Cho H, Yoon SM (2018) Divide and conquer-based 1D CNN human activity recognition using test data sharpening. Sensors 18(4):1055 Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Conference on empirical methods in natural language processing (EMNLP 2014), Doha, Qatar, pp 1724–1734 Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS 2014 workshop on deep learning Deng J, Guo J, Xue N, Zafeiriou S (2019) Arcface: additive angular margin loss for deep face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4690–4699 Dobbin KK, Simon RM (2011) Optimally splitting cases for training and testing high dimensional classifiers. BMC Med Genom 4(1):31 Ferrari A, Micucci D, Mobilio M, Napoletano P (2020) On the personalization of classification models for human activity recognition. IEEE Access 8:32066–32079 Foerster F, Smeja M, Fahrenberg J (1999) Detection of posture and motion by accelerometry: a validation study in ambulatory monitoring. Comput Hum Behav 15(5):571–583 Gao W, Zhang L, Teng Q, Wu H, Min F, He J (2020) DanHAR: dual attention network for multimodal human activity recognition using wearable sensors. Gers FA, Schraudolph NN, Schmidhuber J (2002) Learning precise timing with LSTM recurrent networks. J Mach Learn Res 3:115–143 Graves A, Liwicki M, Fernández S, Bertolami R, Bunke H, Schmidhuber J (2008) A novel connectionist system for unconstrained handwriting recognition. IEEE Trans Pattern Anal Mach Intell 31(5):855–868 Gutchess D, Checka N, Snorrason MS (2007) Learning patterns of human activity for anomaly detection. Intelligent computing: theory and applications. International Society for Optics and Photonics, Washington, p 65600Y Hammerla NY, Halloran S, Plötz T (2016) Deep, convolutional, and recurrent models for human activity recognition using wearables. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, pp 1533–1540 Hassan MM, Uddin MZ, Mohamed A, Almogren A (2018) A robust human activity recognition system using smartphone sensors and deep learning. Future Gener Comput Syst 81:307–313 Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554 Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780 Husken M, Stagge P (2003) Recurrent neural networks for time series classification. Neurocomputing 50:223–235 Hyndman R, Koehler AB, Ord JK, Snyder RD (2008) Forecasting with exponential smoothing: the state space approach. Springer Science and Business Media, Cham Jalal A, Kamal S, Kim D (2014) A depth video sensor-based life-logging human activity recognition system for elderly care in smart indoor environments. Sensors 14(7):11735–11759 Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In: Proceedings of the 23rd ACM international conference on Multimedia, pp 1307–1310 Johnson RA, Miller I, Freund JE (2000) Probability and statistics for engineers. Pearson Education, London, p 642 Jordao A, Kloss R, Schwartz WR (2018) Latent HyperNet: exploring the layers of convolutional neural networks. In: 2018 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1–7 Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980 Kolekar MH, Dash DP (2016) Hidden markov model based human activity recognition using shape and optical flow based features. In: 2016 IEEE Region 10 Conference (TENCON), IEEE, pp 393–397 Kwapisz JR, Weiss GM, Moore SA (2011) Activity recognition using cell phone accelerometers. ACM SIGKDD Explor Newsl 12(2):74–82 Li H, Trocan M (2019) Deep learning of smartphone sensor data for personal health assistance. Microelectron J 88:164–172 Li F, Shirahama K, Nisar MA, Köping L, Grzegorzek M (2018) Comparison of feature learning methods for human activity recognition using wearable sensors. Sensors 18(2):679 Liciotti D, Bernardini M, Romeo L, Frontoni E (2020) A sequential deep learning application for recognising human activities in smart homes. Neurocomputing 396:501–513 Lv T, Wang X, Jin L, Xiao Y, Song M (2020) Margin-based deep learning networks for human activity recognition. Sensors 20(7):1871 Mehdiyev N, Lahann J, Emrich A, Enke D, Fettke P, Loos P (2017) Time series classification using deep learning for process planning: a case from the process industry. Proced Comput Sci 114:242–249 Micucci D, Mobilio M, Napoletano P (2017) Unimib shar: A dataset for human activity recognition using acceleration data from smartphones. Appl Sci 7(10):1101 Mozer MC (1998) The neural network house: an environment hat adapts to its inhabitants. In: Proceedings of AAAI Spring Symposium of Intelligent Environments. Mukherjee D, Mondal R, Singh PK, Sarkar R, Bhattacharjee D (2020) EnsemConvNet: a deep learning approach for human activity recognition using smartphone sensors for healthcare applications. Multimed Tools Appl 79(41):31663–31690 Murad A, Pyun JY (2017) Deep recurrent neural networks for human activity recognition. Sensors 17(11):2556 Nweke HF, Teh YW, Al-Garadi MA, Alo UR (2018) Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: state of the art and research challenges. Expert Syst Appl 105:233–261 Oniga S, Sütő J (2014) Human activity recognition using neural networks. In: Proceedings of the 2014 15th International Carpathian Control Conference (ICCC), IEEE, pp 403–406 Osmani V, Balasubramaniam S, Botvich D (2008) Human activity recognition in pervasive health-care: supporting efficient remote collaboration. J Netw Comput Appl 31(4):628–655 Paul P, George T (2015) An effective approach for human activity recognition on smartphone. In: 2015 IEEE International Conference on Engineering and Technology (ICETECH), IEEE, pp 1–3 Plötz T, Hammerla NY, Olivier P (2011) Feature learning for activity recognition in ubiquitous computing. In: 22nd international joint conference on artificial intelligence, IJCAI 2011, pp 1729–1734 Powell HC, Hanson MA, Lach J (2007) A wearable inertial sensing technology for clinical assessment of tremor. In: 2007 IEEE Biomedical Circuits and Systems Conference, IEEE, pp 9–12 Ramasamy SR, Roy N (2018) Recent trends in machine learning for human activity recognition—a survey. Wiley Interdiscip Rev 8(4):e1254 Ravi D, Wong C, Lo B, Yang GZ (2016) A deep learning approach to on-node sensor data analytics for mobile or wearable devices. IEEE J Biomed Health Inform 21(1):56–64 Ravuri S, Stolcke A (2015) Recurrent neural network and LSTM models for lexical utterance classification. In: Sixteenth Annual Conference of the International Speech Communication Association. Ronao CA, Cho SB (2016) Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst Appl 59:235–244 Sabatini AM, Martelloni C, Scapellato S, Cavallo F (2005) Assessment of walking features from foot inertial sensing. IEEE Trans Biomed Eng 52(3):486–494 Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681 Sefen B, Baumbach S, Dengel A, Abdennadher S (2016) Human activity recognition. In: Proceedings of the 8th International Conference on Agents and Artificial Intelligence. SCITEPRESS-Science and Technology Publications, Lda, pp 488–493 Shen G, Tan Q, Zhang H, Zeng P, Xu J (2018) Deep learning with gated recurrent unit networks for financial sequence predictions. Proced Comput Sci 131:895–903 Simonyan K, Zisserman A (2014) Two-stream convolutional networks for action recognition in videos. Advances in neural information processing systems. Springer, Cham, pp 568–576 Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958 Sukor AA, Zakaria A, Rahim NA (2018) Activity recognition using accelerometer sensor and machine learning classifiers. In: 2018 IEEE 14th International Colloquium on Signal Processing and its Applications (CSPA), IEEE, pp 233–238 Teng Q, Wang K, Zhang L, He J (2020) The layer-wise training convolutional neural networks using local loss for sensor-based human activity recognition. IEEE Sens J 20(13):7265–7274 Torres-Huitzil C, Alvarez-Landero A (2015) Accelerometer-based human activity recognition in smartphones for healthcare services. Mobile health. Springer, Cham, pp 147–169 Uddin MZ, Hassan MM, Almogren A, Zuair M, Fortino G, Torresen J (2017) A facial expression recognition system using robust face features from depth videos and deep learning. Comput Electr Eng 63:114–125 Uddin MZ, Hassan MM, Alsanad A, Savaglio C (2020) A body sensor data fusion and deep recurrent neural network-based behavior recognition approach for robust healthcare. Inform Fus 55:105–115 Ullah M, Ullah H, Khan SD, Cheikh FA (2019) Stacked Lstm network for human activity recognition using smartphone data. In: 2019 8th European Workshop on Visual Information Processing (EUVIP), IEEE, pp 175–180 Veeriah V, Zhuang N, Qi GJ (2015) Differential recurrent neural networks for action recognition. In: Proceedings of the IEEE international conference on computer vision, pp 4041–4049 Vepakomma P, De D, Das SK, Bhansali S (2015) A-Wristocracy: Deep learning on wrist-worn sensing for recognition of user complex activities. In: 2015 IEEE 12th International conference on wearable and implantable body sensor networks (BSN), IEEE, pp 1–6 Vu TH, Dang A, Dung L, Wang JC (2017) Self-gated recurrent neural networks for human activity recognition on wearable devices. In: Proceedings of the on Thematic Workshops of ACM Multimedia 2017, pp 179–185 Wang J, Chen Y, Hao S, Peng X, Hu L (2019) Deep learning for sensor-based activity recognition: a survey. Pattern Recogn Lett 119:3–11 Woodruff RB, Gardial SF (1996) Know your customer: new approaches to customer value and satisfaction blackwell. Cambridge University, Cambridge Woznowski P, King R, Harwin W, Craddock I (2016) A human activity recognition framework for healthcare applications: ontology, labelling strategies, and best practice. In: IoTBD, pp 369–377 Wu GE, Xue S (2008) Portable preimpact fall detector with inertial sensors. IEEE Trans Neural Syst Rehabil Eng 16(2):178–183 Xia K, Huang J, Wang H (2020) LSTM-CNN architecture for human activity recognition. IEEE Access 8:56855–56866 Yager RR (2008) Time series smoothing and OWA aggregation. IEEE Trans Fuzzy Syst 16(4):994–1007 Zahin A, Hu RQ (2019) Sensor-based human activity recognition for smart healthcare: a semi-supervised machine learning. International conference on artificial intelligence for communications and networks. Springer, Cham, pp 450–472 Zainudin MS, Sulaiman MN, Mustapha N, Perumal T (2015) Activity recognition based on accelerometer sensor using combinational classifiers. In: 2015 IEEE Conference on Open Systems (Icos), IEEE, pp 68–73 Zeng M, Nguyen LT, Yu B, Mengshoel OJ, Zhu J, Wu P, Zhang J (2014) Convolutional neural networks for human activity recognition using mobile sensors. In: 6th International Conference on Mobile Computing, Applications and Services, IEEE, pp 197–205