An Efficient Human Computer Interaction through Hand Gesture Using Deep Convolutional Neural Network
Tóm tắt
This paper focuses on the achievement of effective human–computer interaction using only webcam by continuous locating or tracking and recognizing the hand region. We detected the region of interest (ROI) in the captured image range and classify hand gestures for specific tasks. Firstly, background subtraction is used based on the main frame captured by webcam, and some preprocessing are done, and then YCrCb skin segmentation is used on RGB subtracted image. The ROI is detected using Haar cascade classifier for hand palm detection. Next, kernelized correlation filters tracking algorithm is used to avoid noise or background influences for tracking the ROI, and the median-flow tracking algorithm is used for depth tracking. The ROI is converted to a binary channel (black and white), resized to 54 × 54. Then gesture recognition is done using a 2D convolutional neural network (CNN) by entering the preprocessed ROI on the architecture. Two predictions are made based on skin segmented frame and image dilated frame, and gesture is recognized from the maximum value of those two predictions. The tracking and recognition process is continued until the ROI is presented on the frames. Finally, after validation, the proposed system has successfully obtained a recognition rate of 98.44%, which is usable for the practical and real-time application.
Tài liệu tham khảo
G Coleman, R Ward. Gesture recognition performance, applications and features. 4th ed. New York : Nova Science Publishers, Inc; 2018.
A Voulodimos, N Doulamis, A Doulamis, E Protopapadakis. Deep learning for computer vision: a brief review. Comput Intell Neurosci. 2018;2018:1–13. https://doi.org/10.1155/2018/7068349.
Rezwanul Haque M, Milon Islam M, Saeed Alam K, Iqbal H. A computer vision based lane detection approach. Int J Image, Graph Signal Process. 2019;11:27–34. https://doi.org/10.5815/ijigsp.2019.03.04.
Dabre K, Dholay S (2014) Machine learning model for sign language interpretation using webcam images. In: 2014 International conference on circuits, systems, communication and information technology applications, CSCITA 2014
Hasan H, Abdul-Kareem S. Static hand gesture recognition using neural networks Artif. Intell Rev. 2014;41:147–81. https://doi.org/10.1007/s10462-011-9303-1.
Molchanov P, Yang X, Gupta S, Kim K, Tyree S, Kautz J (2016) online detection and classification of dynamic hand gestures with recurrent 3d convolutional neural Networks. In: proceedings of the IEEE computer society conference on computer vision and pattern Recognition
Haria A, Subramanian A, Asokkumar N, Poddar S, Nayak JS. Hand gesture recognition for human computer interaction. Procedia Comp Sci. 2017;115:367–74.
Zhang Z. Microsoft kinect sensor and its effect. IEEE Multimed. 2012;19(2):4–10. https://doi.org/10.1109/MMUL.2012.24.
Memo A, Minto L, Zanuttigh P (2015) Exploiting silhouette descriptors and synthetic data for hand gesture recognition. In: Italian chapter conference 2015—smart tools and apps in computer graphics, STAG 2015
Keskin C, Kiraç F, Kara YE, Akarun L (2011) Real time hand pose estimation using depth sensors. In: proceedings of the IEEE international conference on computer Vision
Rigatti SJ. Random Forest J Insur Med. 2017. https://doi.org/10.17849/insm-47-01-31-39.1.
Rahman MA, Sadi MS, Islam MM, Saha P (2019) Design and Development of Navigation Guide for Visually Impaired People. In: 2019 IEEE International conference on biomedical engineering, computer and information technology for health (BECITHCON). IEEE, pp 89–92
Khanom M, Sadi MS, Islam MM (2019) A comparative study of walking assistance Tools developed for the visually impaired people. 1st Int conf adv sci eng robot technol 2019, ICASERT 2019 2019:1–5. https://doi.org/10.1109/ICASERT.2019.8934566
Rahman MM, Islam M, Ahmmed S. “BlindShoe”: an electronic guidance system for the visually impaired people. J Telecommun Electron Comp Eng (JTEC). 2019;11:49–54.
Habib A, Islam MM, Kabir MN, Mredul MB, Hasan M. Staircase detection to guide visually impaired people: a hybrid approach. Rev d’Intell Artif. 2019;33:327–34. https://doi.org/10.18280/ria.330501.
Islam MM, Sadi MS, Zamli KZ, Ahmed MM. Developing walking assistants for visually impaired people: a review. IEEE Sens J. 2019;19:2814–28. https://doi.org/10.1109/JSEN.2018.2890423.
Alam N, Islam M, Habib A, Mredul MB. Staircase detection systems for the visually impaired people : a review. Int J Comp Sci Inf Secur (IJCSIS). 2018;16:13–18.
Kamal MM, Bayazid AI, Sadi MS, Islam MM, Hasan N (2017) Towards developing walking assistants for the visually impaired people. In: 2017 IEEE region 10 humanitarian technology Conference (R10-HTC). IEEE, pp 238–241
Islam MM, Sadi MS, Islam MM, Hasan MK (2018) A New Method for Road Surface Detection. In: 2018 4th International conference on electrical engineering and information & communication technology (iCEEiCT). IEEE, pp 624–629
Islam MM, Sadi MS (2018) Path Hole Detection to Assist the Visually Impaired People in Navigation. In: 2018 4th International conference on electrical engineering and information & communication technology (iCEEiCT). IEEE, pp 268–273
Islam MM, Neom NH, Imtiaz MS, Nooruddin S, Islam MR, Islam MR. A review on fall detection systems using data from smartphone sensors. Ing des Syst d’Inf. 2019;24:569–76. https://doi.org/10.18280/isi.240602.
Islam MM, Hasan MK, Billah MM, Uddin MM (2017) Development of smartphone-based student attendance system. In: 2017 IEEE region 10 humanitarian technology conference (R10-HTC). IEEE, pp 230–233
Nooruddin S, Milon Islam M, Sharna FA. An IoT based device-type invariant fall detection system. Internet Things. 2020;9:100130. https://doi.org/10.1016/j.iot.2019.100130.
Rahaman A, Islam M, Islam M, Sadi M, Nooruddin S. Developing IoT based smart health monitoring systems: a review. Rev d’Int Artif. 2019;33:435–40. https://doi.org/10.18280/ria.330605.
Islam MM, Rahaman A, Islam MR. Development of smart healthcare monitoring system in IoT environment. SN Comput Sci. 2020;1:185. https://doi.org/10.1007/s42979-020-00195-y.
Brutzer S, Höferlin B, Heidemann G (2011) Evaluation of background subtraction techniques for video surveillance. In: proceedings of the IEEE computer society conference on computer vision and pattern recognition
Shaik KB, Ganesan P, Kalist V, Sathish B, Jenitha JMM. Comparative study of skin color detection and segmentation in HSV and YCbCr color space. Procedia Comp Sci. 2015;57:41–8. https://doi.org/10.1016/j.procs.2015.07.362.
Tsagaris A, Manitsaris S. Colour space comparison for skin detection in finger gesture recognition. Int J Adv Eng Technol. 2013;6(4):1431.
Soo S. Object detection using Haar-cascade Classifier. Inst Comp Sci Univ Tartu. 2014;2(3):1–12.
Fisher R, Perkins S, Walker A, Wolfart E (2003) Spatial Filters—Gaussian Smoothing. Image Process. Learn. Resour.
Tang M, Yu B, Zhang F, Wang J (2018) High-speed tracking with Multi-kernel correlation Filters. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition
Grigorev A, Derevitskii I, Bochenina K. Analysis of special transport behavior using computer vision analysis of video from traffic cameras. Commun Comput Inform Sci. 2018;858:289–301. https://doi.org/10.1007/978-3-030-02843-5_23.
Cheng C, Parhi KK. Fast 2D convolution algorithms for convolutional neural networks. IEEE Trans Circuits Syst I Regul Pap. 2020. https://doi.org/10.1109/TCSI.2020.2964748.
Lecun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–44.
Haque S, Sadi MS, Rafi MEH, Islam MM, Hasan MK (2020) Real-Time Crowd Detection to Prevent Stampede. pp 665–678
Muhammad LJ, Islam MM, Sharif US, Ayon SI. Predictive data mining models for novel coronavirus (COVID-19) Infected Patients Recovery. SN Comput Sci. 2020;1:216. https://doi.org/10.1007/s42979-020-00216-w.
Hasan MK, Islam MM, Hashem MMA (2016) Mathematical model development to detect breast cancer using multigene genetic programming. In: 2016 5th International conference on informatics, electronics and vision (ICIEV). IEEE, pp 574–579
Das S, Sadi MS, Ahsanul Haque M, Islam MM (2019) A machine learning approach to protect electronic devices from damage using the concept of outlier. In: 2019 1st international conference on advances in science, engineering and robotics technology (ICASERT). IEEE, pp 1–6
Haque MR, Islam MM, Iqbal H, Reza MS, Hasan MK (2018) Performance evaluation of random forests and artificial neural networks for the classification of Liver Disorder. In: 2018 International conference on computer, communication, chemical, material and electronic engineering (IC4ME2). IEEE, pp 1–5
Islam MM, Iqbal H, Haque MR, Hasan MK (2017) Prediction of breast cancer using support vector machine and K-nearest neighbors. In: 2017 IEEE Region 10 Humanitarian technology conference (R10-HTC). IEEE, pp 226–229
Islam Ayon S, Milon Islam M. Diabetes prediction: a deep learning approach. Int J Inf Eng Electron Bus. 2019;11:21–7. https://doi.org/10.5815/ijieeb.2019.02.03.
Milon Islam M, Kabir MN, Sadi MS, Morsalin MI, Haque A, Wang J. A novel approach towards tamper detection of digital holy quran generation. Lect Notes Electr Eng. 2020;632:297–308.
Ayon SI, Islam MM, Hossain MR. Coronary artery heart disease prediction: a comparative study of computational intelligence techniques. IETE J Res. 2020. https://doi.org/10.1080/03772063.2020.1713916.
Hasan M, Islam MM, Zarif MII, Hashem MMA. Attack and anomaly detection in IoT sensors in IoT sites using machine learning approaches. Internet Things. 2019;7:100059. https://doi.org/10.1016/j.iot.2019.100059.