Facial expression recognition of intercepted video sequences based on feature point movement trend and feature block texture variation

Ekman, 1971, Constants across cultures in the face and emotion, J. Personal. Soc. Psychol., 17, 124, 10.1037/h0030377 Wold, 1987, Principal component analysis, Chemom. Intel. Lab. Syst., 2, 37, 10.1016/0169-7439(87)80084-9 Y. Zhu, X. Li, G. Wu, Face expression recognition based on equable principal component analysis and linear regression classification, in: The 3rd IEEE International Conference on Systems and Informatics, ICSAI, Shanghai, China, 19–21 Nov. 2016, pp. 876–880, http://dx.doi.org/10.1109/ICSAI.2016.7811074. Roweis, 2000, Nonlinear dimensionality reduction by locally linear embedding, Science, 290, 2323, 10.1126/science.290.5500.2323 D. Meng, G. Cao, Z. He, W. Cao, Facial expression recognition based on LLENet, in: IEEE International Conference on Bioinformatics and Biomedicine, BIBM, Shenzhen, China, 15–18 Dec. 2016, pp. 1915–1917, http://dx.doi.org/10.1109/BIBM.2016.7822814. Wang, 2016, Facial expression recognition using sparse local Fisher discriminant analysis, Neurocomputing, 174, 756, 10.1016/j.neucom.2015.09.083 Siddiqi, 2015, Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields, IEEE Trans. Image Process., 24, 1386, 10.1109/TIP.2015.2405346 S.Z. Li, X.W. Hou, H.J. Zhang, Q.S. Cheng, Learning spatially localized, parts-based representation, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, Kauai, HI, USA 8–14 Dec. 2001, pp. I–I, http://dx.doi.org/10.1109/CVPR.2001.990477. Zheng, 2017, Micro-expression recognition based on 2d gabor filter and sparse representation, J. Phys.: Conf. Ser., 787 Chao, 2015, Facial expression recognition based on improved local binary pattern and class-regularized locality preserving projection, Signal Process., 117, 1, 10.1016/j.sigpro.2015.04.007 Wang, 2015, Efficient spatio-temporal local binary patterns for spontaneous facial micro-expression recognition, PloS One, 10 Happy, 2015, Automatic facial expression recognition using features of salient facial patches, IEEE Trans. Affect. Comput., 6, 1, 10.1109/TAFFC.2014.2386334 Wang, 2017, Expression-assisted facial action unit recognition under incomplete AU annotation, Pattern Recognit., 61, 78, 10.1016/j.patcog.2016.07.028 Chu, 2016, Selective transfer machine for personalized facial expression analysis, IEEE Trans. Pattern Anal. Mach. Intel., 39, 529, 10.1109/TPAMI.2016.2547397 Yi, 2014, Facial expression recognition considering individual differences in facial structure and texture, IET Comput. Vis., 8, 429, 10.1049/iet-cvi.2013.0171 Yi, 2013, Facial expression recognition based on feature point vector and texture deformation energy parameters, J. Electron. Inf. Technol., 35, 2403, 10.3724/SP.J.1146.2012.01569 Qiu, 2016, Facial expression recognition using temporal relations among facial movements, Acta Electron. Sin., 44, 1307 Shao, 2015, Spontaneous facial expression recognition based on rgb-d dynamic sequences, J. Comput.-Aided Des. Comput. Graph., 27, 847 Niu, 2005, Extraction of key expression frame in video sequence, J. Computex Res. Dev., 42, 264 Ye, 2005 Liu, 2016, Learning expressionlets via universal manifold model for dynamic facial expression recognition, IEEE Trans. Image Process., 25, 5920, 10.1109/TIP.2016.2615424 Yan, 2018, Collaborative discriminative multi-metric learning for facial expression recognition in video, Pattern Recognit., 75, 33, 10.1016/j.patcog.2017.02.031 Fan, 2017, A dynamic framework based on local Zernike moment and motion history image for facial expression recognition, Pattern Recognit., 64, 399, 10.1016/j.patcog.2016.12.002 Zhao, 2016, Joint patch and multi-label learning for facial action unit and holistic expression recognition, IEEE Trans. Image Process., 25, 3931, 10.1109/TIP.2016.2570550 Huang, 2017, Discriminative spatiotemporal local binary pattern with revisited integral projection for spontaneous facial micro-expression recognition, IEEE Trans. Affect. Comput., 14, 1 Zhang, 2017, Facial expression recognition based on deep evolutional spatial–temporal networks, IEEE Trans. Image Process., 26, 4193, 10.1109/TIP.2017.2689999 Fang, 2014, Facial expression recognition in dynamic sequences: An integrated approach, Pattern Recognit., 47, 1271, 10.1016/j.patcog.2013.09.023 Majumder, 2018, Automatic facial expression recognition system using deep network-based data fusion, IEEE Trans. Cybern., 48, 103, 10.1109/TCYB.2016.2625419 Barman, 2019, Facial expression recognition using distance and texture signature relevant features, Appl. Soft Comput., 77, 88, 10.1016/j.asoc.2019.01.011 Fernandez, 2019, FERAtt: Facial Expression Recognition with Attention net, Comput. Vis. Pattern Recognit., 1 Xie, 2019, Facial expression recognition using hierarchical features with deep comprehensive multipatches aggregation convolutional neural networks, IEEE Trans. Multimedia, 21, 211, 10.1109/TMM.2018.2844085 Yi, 2016, Illumination compensation for facial feature point localization in a single 2d face image, Neurocomputing, 173, 573, 10.1016/j.neucom.2015.07.092 Yi, 2015, Illumination normalization of face image based on illuminant direction estimation and improved retinex, PLoS One, 10, 10.1371/journal.pone.0122200 Richhariya, 2019, Facial expression recognition using iterative universum twin support vector machine, Appl. Soft Comput., 76, 53, 10.1016/j.asoc.2018.11.046 Kamarol, 2017, Joint facial expression recognition and intensity estimation based on weighted votes of image sequences, Pattern Recognit. Lett., 92, 25, 10.1016/j.patrec.2017.04.003 Sariyanidi, 2017, Learning bases of activity for facial expression recognition, IEEE Trans. Image Process., 26, 1965, 10.1109/TIP.2017.2662237 Islam, 2017, A robust incremental clustering-based facial feature tracking, Appl. Soft Comput., 53, 34, 10.1016/j.asoc.2016.12.033 Guo, 2016, Dynamic facial expression recognition with atlas construction and sparse representation, IEEE Trans. Image Process., 25, 1977, 10.1109/TIP.2016.2537215 Y.L. Xue, X. Mao, F. Zhang, Beihang university facial expression database and multiple facial expression recognition, in: IEEE International Conference on Machine Learning and Cybernetics, ICMLC, Dalian, China, 13–16 Aug. 2006, pp. 3282–3287, http://dx.doi.org/10.1109/ICMLC.2006.258460. M. Pantic, M. Valstar, R. Rademaker, L. Maat, Web-based database for facial expression analysis, in: IEEE International Conference on Multimedia and Expo, ICME, Amsterdam, Netherlands, 6 Jul. 2005, pp. 317–321,. M. Valstar, M. Pantic, Induced disgust, happiness and surprise: an addition to the mmi facial expression database, in: The 3rd International Workshop on Emotion (satellite of LREC): Corpora for Research on Emotion and Affect, Valletta, Malta, 2010, pp. 65–70. Kim, 2017, Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition, IEEE Trans. Affect. Comput., 99, 1 R. Jensen, C. Cornelis, A new approach to fuzzy-rough nearest neighbour classification, in: The 6th International Conference on Rough Sets and Current Trends in Computin, RSCTC, Akron, OH, USA, 23–25 Oct. 2008, pp. 310–319,. Freund, 1997, A desicion-theoretic generalization of on-line learning and an application to boosting, J. Comput. Syst. Sci., 55, 119, 10.1006/jcss.1997.1504 Platt, 1999, Fast training of support vector machines using sequential minimal optimization, 185