Facial expression recognition based on Local Binary Patterns: A comprehensive study

Pantic, 2000, Automatic analysis of facial expressions: the state of art, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1424, 10.1109/34.895976

Fasel, 2003, Automatic facial expression analysis: a survey, Pattern Recognition, 36, 259, 10.1016/S0031-3203(02)00052-3

M. Pantic, L. Rothkrantz, Toward an affect-sensitive multimodal human–computer interaction, in: Proceeding of the IEEE, vol. 91, 2003, pp. 1370–1390.

Tian, 2005

Yacoob, 1996, Recognizing human facial expression from long image sequences using optical flow, IEEE Transactions on Pattern Analysis and Machine Intelligence, 18, 636, 10.1109/34.506414

Essa, 1997, Coding, analysis, interpretation, and recognition of facial expressions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 757, 10.1109/34.598232

Lyons, 1999, Automatic classification of single facial images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 1357, 10.1109/34.817413

Donato, 1999, Classifying facial actions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 974, 10.1109/34.799905

Pantic, 2000, Expert system for automatic analysis of facial expression, Image and Vision Computing, 18, 881, 10.1016/S0262-8856(00)00034-2

Tian, 2001, Recognizing action units for facial expression analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 97, 10.1109/34.908962

Cohen, 2003, Facial expression recognition from video sequences: temporal and static modeling, Computer Vision and Image Understanding, 91, 160, 10.1016/S1077-3142(03)00081-X

L. Yin, J. Loi, W. Xiong, Facial expression representation and recognition based on texture augmentation and topographic masking, in: ACM Multimedia, 2004.

M. Yeasin, B. Bullot, R. Sharma, From facial expression to level of interests: a spatio-temporal approach, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2004.

J. Hoey, J.J. Little, Value directed learning of gestures and facial displays, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2004.

Y. Chang, C. Hu, M. Turk, Probabilistic expression analysis on manifolds, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2004.

R.E. Kaliouby, P. Robinson, Real-time inference of complex mental states from facial expressions and head gestures, in: IEEE CVPR Workshop on Real-time Vision for Human–Computer Interaction, 2004.

Pantic, 2004, Facial action recognition for facial expression analysis from static face images, IEEE Transactions on Systems, Man, and Cybernetics, 34, 1449, 10.1109/TSMCB.2004.825931

Zhang, 2005, Active and dynamic information fusion for facial expression understanding from image sequences, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 1

M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, J. Movellan, Recognizing facial expression: machine learning and application to spotaneous behavior, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2005.

F. Dornaika, F. Davoine, Simultaneous facial action tracking and expression recognition using a particle filter, in: IEEE International Conference on Computer Vision (ICCV), 2005.

C.S. Lee, A. Elgammal, Facial expression analysis using nonlinear decomposable generative models, in: IEEE International Workshop on Analysis and Modeling of Faces and Gestures (AMFG), 2005.

M. Valstar, I. Patras, M. Pantic, Facial action unit detection using probabilistic actively learned support vector machines on tracked facial point data, in: IEEE Conference on Computer Vision and Pattern Recognition Workshop, vol. 3, 2005, pp. 76–84.

M. Valstar, M. Pantic, Fully automatic facial action unit detection and temporal analysis, in: IEEE Conference on Computer Vision and Pattern Recognition Workshop, 2006, p. 149.

Pantic, 2006, Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences, IEEE Transactions on Systems, Man, and Cybernetics, 36, 433, 10.1109/TSMCB.2005.859075

Z. Zhang, M.J. Lyons, M. Schuster, S. Akamatsu, Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron, in: IEEE International Conference on Automatic Face & Gesture Recognition (FG), 1998.

Y. Tian, Evaluation of face resolution for expression analysis, in: CVPR Workshop on Face Processing in Video, 2004.

Ojala, 1996, A comparative study of texture measures with classification based on featured distribution, Pattern Recognition, 29, 51, 10.1016/0031-3203(95)00067-4

Ojala, 2002, Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 971, 10.1109/TPAMI.2002.1017623

T. Ahonen, A. Hadid, M. Pietikäinen, Face recognition with local binary patterns, in: European Conference on Computer Vision (ECCV), 2004.

A. Hadid, M. Pietikäinen, T. Ahonen, A discriminative feature space for detecting and recognizing faces, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2004.

Feng, 2004, A coarse-to-fine classification scheme for facial expression recognition, vol. 3212, 668

Y. Tian, L. Brown, A. Hampapur, S. Pankanti, A. Senior, R. Bolle, Real world real-time automatic recognition of facial expression, in: IEEE Workshop on Performance Evaluation of Tracking and Surveillance (PETS), Australia, 2003.

C. Shan, S. Gong, P.W. McOwan, Robust facial expression recognition using local binary patterns, in: IEEE International Conference on Image Processing (ICIP), Genoa, vol. 2, 2005, pp. 370–373.

S. Liao, W. Fan, C.S. Chung, D.-Y. Yeung, Facial expression recognition using advanced local binary patterns, tsallis entropies and global appearance features, in: IEEE International Conference on Image Processing (ICIP), 2006, pp. 665–668.

C. Darwin, The Expression of the Emotions in Man and Animals, John Murray, London, 1872.

M. Suwa, N. Sugie, K. Fujimora, A preliminary note on pattern recognition of human emotional expression, in: International Joint Conference on Pattern Recognition, 1978, pp. 408–410.

M. Bartlett, G. Littlewort, C. Lainscsek, I. Fasel, J. Movellan, Machine learning methods for fully automatic recognition of facial expressions and facial actions, in: IEEE International Conference on Systems, Man & Cybernetics, Netherlands, 2004.

M. Turk, A.P. Pentland, Face recognition using eigenfaces, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1991.

Belhumeur, 1997, Eigenfaces vs. fisherfaces: recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 711, 10.1109/34.598228

Bartlett, 2002, Face recognition by independent component analysis, IEEE Transactions on Neural Networks, 13, 1450, 10.1109/TNN.2002.804287

Shan, 2005, Appearance manifold of facial expression, 221

C. Padgett, G. Cottrell, Representing face images for emotion classification, in: Advances in Neural Information Processing Systems (NIPS), 1997.

Ekman, 1978

Ekman, 1976

T. Kanade, J. Cohn, Y. Tian, Comprehensive database for facial expression analysis, in: IEEE International Conference on Automatic Face & Gesture Recognition (FG), 2000.

P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2001.

M.R. Everingham, A. Zisserman, Regression and classification approaches to eye localization in face images, in: IEEE International Conference on Automatic Face & Gesture Recognition (FG), 2006, pp. 441–446.

M. Bartlett, G. Littlewort, I. Fasel, R. Movellan, Real time face detection and facial expression recognition: development and application to human–computer interaction, in: CVPR Workshop on CVPR for HCI, 2003.

Vapnik, 1998

C.-W. Hsu, C.-C. Chang, C.-J. Lin, A Practical Guide to Support Vector Classification, Tech. Rep., Taipei, 2003.

Feng, 2005, Facial expression recognition with local binary patterns and linear programming, Pattern Recognition and Image Analysis, 15, 546

G. Guo, C.R. Dyer, Simultaneous feature selection and classifier training via linear programming: a case study for face expression recognition, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2003.

Schapire, 1999, Improved boosting algorithms using confidence-rated predictions, Maching Learning, 37, 297, 10.1023/A:1007614523901

G. Zhang, X. Huang, S.Z. Li, Y. Wang, X. Wu, Boosting local binary pattern (lbp)-based face recognition, in: Chinese Conference on Biometric Recognition (SINOBIOMETRICS), 2004, pp. 179–186.

C. Shan, S. Gong, P.W. McOwan, Conditional mutual information based boosting for facial expression recognition, in: British Machine Vision Conference (BMVC), Oxford, vol. 1, 2005, pp. 399–408.

Freund, 1997, A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences, 55, 119, 10.1006/jcss.1997.1504

M. Pantic, M. Valstar, R. Rademaker, L. Maat, Web-based database for facial expression analysis, in: IEEE International Conference on Multimedia and Expo (ICME), 2005.

D. Bolme, M. Teixeria, J. Beveridge, B. Draper, The CSU face identification evaluation system: its purpose, features and structure, in: International Conference on Vision Systems, 2003, pp. 304–311.

Littlewort, 2006, Dynamics of facial expression extracted automatically from video, Image and Vision Computing, 24, 615, 10.1016/j.imavis.2005.09.011

Bassili, 1979, Emotion recognition: the role of facial movement and the relative importance of upper and lower area of the face, Journal of Personality and Social Psychology, 37, 2049, 10.1037/0022-3514.37.11.2049

Zhao, 2007, Dynamic texture recognition using local binary patterns with an application to facial expressions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 915, 10.1109/TPAMI.2007.1110