Image based recognition of Pakistan sign language
Tóm tắt
Sign language is the language of gestures used for non-verbal communication. This paper deals with alphabets and digit signs recognition from Pakistan Sign Language (PSL). The deep pixels-based analysis is pursued for the recognition of fingers (from index to small finger) while thumb position is determined through Template Matching. After fingers identification, the isolated signs are recognized based on finger states of being raised or lowered besides thumb’s position in 2D. For a quick recognition, signs are categorized into seven groups. The algorithm identifies these groups following a model of seven phases. The system’s accuracy achieved a satisfactory level of 84.2% when evaluated with signs comprising 180 digits and 240 alphabets.
Tài liệu tham khảo
Al-Jarrah, O. & Alaa, H. 2001. Recognition of gestures in Arabic sign language using neuro-fuzzy systems. Artificial Intelligence, 133(1–2): 117–138.
Alia, O. M., Mandava R., Ramachandram D. & Aziz M. E., 2009. A novel image segmentation algorithm based on harmony fuzzy search algorithm. In international conference of soft computing and pattern recognition, pp 335–340.
Alvi, A. K., Azhar, M. Y. B., Usman, M., Mumtaz, S., Rafiq, S., Rehman, R. U. & Ahmed, I. 2005. Pakistan sign language recognition using statistical template matching. International Conference on Information Technology, (ICIT 2004), Istanbul, Turkey pp. 108–111.
Assaleh K. & Al-Rousan, M. 2005. Recognition of Arabic sign language alphabet using polynomial classifiers. EURASIP Journal on Applied Signal Processing, 13, pp. 2136–2146.
Chalechale A., & Safae, F. 2008. Visual-based interface using hand gesture recognition and object tracking. Iranian Journal of Science and Technology; Transaction B, Engineering Shiraz University, 32: 279–293.
Doliotis, P., Mcmurrough, C., Eckhard, D. & Athitsos, V. 2011. Comparing gesture recognition accuracy using colours and depth information. Conference on Pervasive Technologies Related to Assistive Environments, Heraklion, Crete, Greece.
Erdem, U. M. & Sclaroff, S. 2002. Automatic detection of relevant head gestures in American Sign Language communication. International Conference on Pattern Recognition, Vol.1, BRAC University Dhaka, Bangladesh.
Fan, Z. & Ling S. 2014. Weakly-supervised cross-domain dictionary learning for visual recognition, International Journal of Computer Vision, 109(1–2): 42–59.
Fang, G., Wen, G. & Zhao, D. 2007. Large-vocabulary continuous sign language recognition based on transition-movement models. IEEE Transactions on Systems, Man & Cybernetics, Part A: Systems and Humans, 37(1): 305–314.
Holden, E. & Owens., R. 2001. Visual sign language recognition. Multi-Image Analysis, Lecture Notes in Computer Science, Springer, pp. 270–287.
Jerde, T. E., Soechting, J. E. & Flanders, M. 2003. Biological constraints simplify the recognition of hand shapes. Transactions on Biomedical Engineering, 50(2): 265–269.
Kadous M. W. 1996. Machine recognition of Auslan signs using power gloves towards large-lexicon recognition of sign language. Proceedings of the Workshop on the Integration of Gestures in Language & Speech, pp. 165–174, Delaware and Wilmington, Delaware.
Kuch, J. J. & Huang, T. S. 1995. Vision-based hand modelling and tracking for virtual teleconferencing and tele-collaboration. IEEE International Conference on Computer Vision, Cambridge, MA, USA.
Ling, S., Li, L., & Xuelong, L. 2014. Feature Learning for Image Classification via Multi objective Genetic Programming, IEEE Transactions on Neural Networks and Learning Systems, 25(7): 1359–1371.
Liu, X. & Fujimura, K. 2004. Hand gesture recognition using depth data. In: Proceedings of FGR, Seoul, Korea.
Muir, L. & Leaper, S. 2003. Gaze tracking and its application to video coding for sign language. Picture Coding Symposium, the Robert Gordon University, School hill, Aberdeen, UK.
Nagarajan, S., Subashini, T. & Ramalingam, V. 2012. Vision Based Real Time Finger Counter for Hand Gesture Recognition. International Journal of Technology, 2(2): 1–5, CPMR–IJT.
Ong, E. J., Cooper, H., Pugeault, N. & Bowden, R. 2012. Sign language recognition using sequential pattern trees. In International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2200–2207.
Raees, M., Sehat U. 2014. Alphabet signs recognition using pixels-based analysis. 5th Conference on Language and Technology (CLT14), DHA Suffa University (DSU), Karachi
Rajeshree, R. & Manesh, K. 2009. Gesture recognition by thinning method. International Conference on Digital Image Processing, IEEE Computer Society, Bangkok, pp. 284–287.
Ravikiran, J., Kavi, M., Suhas, M., Dheeraj, R., Sudheender, S. & Nitin, V. P. 2009. Finger detection for sign language recognition. Proceedings of the International Multi Conference of Engineers and Computer Scientists, HongKong
Saad, A., Jonas, B. & Hedvig, K. 2012. Visual recognition of isolated Swedish sign language signs. PhD Thesis Cornell University Ithaca, New York, United States.
Shanableh, T. 2007. Arabic sign language recognition in user independent mode. IEEE International Conference on Intelligent and Advanced Systems, Kuala Lumpur, Malaysia.
Simon, L., Marco, D. & Block, B. 2011. Sign language recognition with kinect. MS Thesis, Freie Universität Berlin, Germany.
Starner, T., Weaver, J. & Pent I. A. 1998. Real-time American sign language recognition using desk and wearable computer based video. IEEE Transactions on Pattern Analysis & Machine Intelligence, 20(2): 1371–1375.
Subha, R. & Balakrishnan, G. 2011. Real-time Indian sign language recognition system to aid deafdumb people. IEEE 13th International Conference on Communication Technology. Indian Institute of Technology Kharagpur, Kharagpur, India.
Sumera, K., Younus, M. & Shaleeza, S. 2008. Recognition of gestures in Pakistani sign language using fuzzy classifier. 8th International Conference on Signal Processing, Computational Geometry and Artificial Vision, (ISCGAV’08), Rhodes, Greece.
Vezhnevets, S. V. & Reeva, A. 2003. A Survey on pixel-based skin colour detection techniques. Proceedings of Graphicon, Moscow, Russia.
Von, A., Knorr, M., Kraiss, K. & Jonghwa, K. 2008. The significance of facial features for automatic sign language recognition. 8th IEEE International Conference on Automatic Face Gesture Recognition, Amsterdam, Netherlands.
Wen, G., Jiyong, M., Jangqin, W. & Chunli, W. 2000. Sign language recognition based on HMM/ANN/DP. International Journal of Pattern Recognition and Artificial Intelligence, 14(5): 587–602.
Yang H. D. & Lee S. W. 2010. Robust sign language recognition with hierarchical conditional random fields. In 20th International Conference on Pattern Recognition, Istanbul Turkey.
Zabulis, X., Baltzakisy, H. & Argyroszy, A. 2007. Vision-based hand & gesture recognition for humancomputer interaction. Institute of Computer Science Foundation for Research and Technology, University of Crete Heraklion, Crete, Hellas, Heraklion.
Zhang, L., Zhen, X., & Shao, L. 2014. Learning object-to-class kernels for scene classification. IEEE Transactions on Image Processing, 23(8): 3241–3253.