Fully automated age-weighted expression classification using real and apparent age
Tóm tắt
After decades of research, automatic facial expression recognition (AFER) has been shown to work well when restricted to subjects with a limited range of ages. Expression recognition in subjects having a large range of ages is harder as it has been shown that ageing, health, and lifestyle affect facial expression. In this paper, we present a discriminative system that explicitly predicts expression across a large range of ages, which we show to perform better than an equivalent system which ignores age. In our system, we first build a fully automatic facial feature point detector (FFPD) using random forest regression voting in a constrained local mode (RFRV-CLM) framework (Cootes et al., in: European conference on computer vision, Springer, Berlin, 2012) which we use to automatically detect the location of key facial points, study the effect of ageing on the accuracy of point localization task. Second, a set of age group estimator and age-specific expression recognizers are trained from the extracted features that include shape, texture, appearance and a fusion of shape with texture, to analyse the effect of ageing on the face features and subsequently on the performance of AFER. We then propose a simple and effective method to recognize the expression across a large range of ages through using a weighted combination rule of a set of age group estimator and age specific expression recognizers (one for each age group), where the age information is used as prior knowledge to the expression classification. The advantage of using the weighted combination of all the classifiers is that more information about the classification can be obtained and subjects whose apparent age puts them in the wrong chronological age group will be dealt with more effectively. The performance of the proposed system was evaluated using three age-expression databases of static and dynamic images for deliberate and spontaneous expressions: FACES (Ebner et al., in Behav Res Methods 42:351–362, 2010) (2052 images), Lifespan (Minear and Park in Behav Res Methods Instrum Comput 36:630–633, 2004) (844 images) and NEMO (Dibeklioğlu et al., in: European conference on computer vision, Springer, Berlin, 2012) (1,243 videos). The results show the system to be accurate and robust against a wide variety of expressions and the age of the subject. Evaluation of point localization, age group estimation and expression recognition against ground truth data was obtained and compared with the existing results of alternative approaches tested on the same data. The quantitative results with 2.1% error rates (using manual points) and 3.0% error rates (fully automatic) of expression classification demonstrated that the results of our novel system were encouraging in comparison with the state-of-the-art systems which ignore age and alternative models recently applied to the problem.
Tài liệu tham khảo
Adolphs R, Tranel D (2004) Impaired judgments of sadness but not happiness following bilateral amygdala damage. J Cogn Neurosci 16(3):453–462
Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. Comput Vision-ECCV 2004:469–481
Alnajar F, Lou Z, Álvarez JM, Gevers T, et al (2014) Expression-invariant age estimation. In: BMVC
Antipov G, Baccouche M, Berrani SA, Dugelay JL (2016) Apparent age estimation from face images combining general and children-specialized deep learning models. In: proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 96–104
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Bromiley P, Adams J, Cootes T (2015) Localisation of vertebrae on dxa images using constrained local models with random forest regression voting. In: Yao J, Glocker B, Klinder T, Li S (eds) Recent advances in computational methods and clinical applications for spine imaging. Springer, Cham, pp 159–171
Bromiley PA, Adams JE, Cootes TF (2015) Automatic localisation of vertebrae in dxa images using random forest regression voting. In: international workshop on computational methods and clinical applications for spine imaging, pp. 38–51. Springer
Bromiley PA, Kariki EP, Adams JE, Cootes TF (2016) Fully automatic localisation of vertebrae in ct images using random forest regression voting. In: international workshop on computational methods and clinical applications for spine imaging, pp. 51–63. Springer
Clapés A, Bilici O, Temirova D, Avots E, Anbarjafari G, Escalera S (2018) From apparent to real age: gender, age, ethnic, makeup, and expression bias analysis in real age estimation. In: proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 2373–2382
Cohn JF, Kruez TS, Matthews I, Yang Y, Nguyen MH, Padilla MT, Zhou F, De la Torre F (2009) Detecting depression from facial actions and vocal prosody. In: 3rd international conference on affective computing and intelligent interaction and workshops, ACII 2009. pp. 1–7. IEEE
Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685
Cootes TF, Ionita MC, Lindner C, Sauer P (2012) Robust and accurate shape model fitting using random forest regression voting. In: European conference on computer vision, pp. 278–291. Springer
Dibeklioğlu H, Salah AA, Gevers T (2012) Are you really smiling at me? spontaneous versus posed enjoyment smiles. In: European conference on computer vision, pp. 525–538. Springer
Dibeklioğlu H, Salah AA, Gevers T (2015) Recognition of genuine smiles. IEEE Trans Multimed 17(3):279–294
Dodgson NA (2004) Variation and extrema of human interpupillary distance. In: stereoscopic displays and virtual reality systems XI, vol. 5291, pp. 36–47. International society for optics and photonics
Ebner NC, Johnson MK (2010) Age-group differences in interference from young and older emotional faces. Cognit Emotion 24(7):1095–1116
Ebner NC, Riediger M, Lindenberger U (2010) Faces-a database of facial expressions in young, middle-aged, and older women and men: development and validation. Beh Res Methods 42(1):351–362
Fölster M, Hess U, Werheid K (2014) Facial age affects emotional expression decoding. Front Psychol 5:30
Goldman AI, Sripada CS (2005) Simulationist models of face-based emotion recognition. Cognition 94(3):193–213
Guo G, Guo R, Li X (2013) Facial expression recognition influenced by human aging. IEEE Trans Affect Comput 4(3):291–298
Hess U, Adams RB, Simard A, Stevenson MT, Kleck RE (2012) Smiling and sad wrinkles: age-related changes in the face and the perception of emotions and intentions. J Exp Soc Psychol 48(6):1377–1380
Houstis O, Kiliaridis S (2009) Gender and age differences in facial expressions. Eur J Orthodont 31(5):459–466
Huo Z, Yang X, Xing C, Zhou Y, Hou P, Lv J, Geng X (2016) Deep age distribution learning for apparent age estimation. In: proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 17–24
Kanade T, Cohn JF, Tian Y (2000) Comprehensive database for facial expression analysis. In: proceedings fourth IEEE international conference on automatic face and gesture recognition (Cat. No. PR00580), pp. 46–53. IEEE
Lawrence AD, Calder AJ (2004) Homologizing human emotions
Lindner C, Bromiley PA, Ionita MC, Cootes TF (2015) Robust and accurate shape model matching using random forest regression-voting. IEEE Trans Pattern Anal Mach Intell 37(9):1862–1874
Lindner C, Cootes T (2015) Fully automatic cephalometric evaluation using random forest regression-voting. In: proceedings of the IEEE international symposium on biomedical imaging (ISBI) 2015–grand challenges in dental X-ray image analysis–automated detection and analysis for diagnosis in cephalometric X-ray image. Citeseer
Lindner C, Thiagarajah S, Wilkinson J, Consortium T (2013) Fully automatic segmentation of the proximal femur using random forest regression voting. IEEE Trans Med Imaging 32(8):1462–1472
Lou Z, Alnajar F, Alvarez JM, Hu N, Gevers T (2018) Expression-invariant age estimation using structured learning. IEEE Trans Pattern Anal Mach Intell 40(2):365–375
Lyons M, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with gabor wavelets. In: proceedings third IEEE international conference on automatic face and gesture recognition, 1998, pp. 200–205. IEEE
Martinez B, Valstar MF (2016) Advances, challenges, and opportunities in automatic facial expression recognition. In: Kawulok M, Celebi E, Smolka B (eds) Advances in face detection and facial image analysis. Springer, Berlin, pp 63–100
Mary R, Jayakumar T (2016) A review on how human aging influences facial expression recognition (fer). In: Abraham A, Haqiq A, Muda AK, Gandhi N (eds) Innovations in bio-inspired computing and applications. Springer, Berlin, pp 313–322
Minear M, Park DC (2004) A lifespan database of adult facial stimuli. Beh Res Methods Instruments Comput 36(4):630–633
Nguyen DT, Cho SR, Shin KY, Bang JW, Park KR (2014) Comparative study of human age estimation with or without preclassification of gender and facial expression. Sci World J. https://doi.org/10.1155/2014/905269
Phillips LH, Allen R (2004) Adult aging and the perceived intensity of emotions in faces and stories. Aging Clin Exp Res 16(3):190–199
Prkachin KM, Solomon PE (2008) The structure, reliability and validity of pain expression: evidence from patients with shoulder pain. Pain 139(2):267–274
Rothe R, Timofte R, Van Gool L (2015) Dex: deep expectation of apparent age from a single image. In: proceedings of the IEEE international conference on computer vision workshops, pp. 10–15
Sariyanidi E, Gunes H, Cavallaro A (2015) Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE Trans Pattern Anal Mach Intell 37(6):1113–1133
Sawant MM, Bhurchandi KM (2019) Age invariant face recognition: a survey on facial aging databases, techniques and effect of aging. Artif Intell Rev 52(2):981–1008
Uřičař M, Timofte R, Rothe R, Matas J, et al (2016) Structured output svm prediction of apparent age, gender and smile from deep features. In: proceedings of the 29th IEEE conference on computer vision and pattern recognision workshop (CVPRW 2016), pp. 730–738. IEEE
Vural E, Cetin M, Ercil A, Littlewort G, Bartlett M, Movellan J (2007) Drowsy driver detection through facial movement analysis. In: international workshop on human-computer interaction, pp. 6–18. Springer
Wang S, Wu S, Gao Z, Ji Q (2016) Facial expression recognition through modeling age-related spatial patterns. Multimed Tools Appl 75(7):3937–3954
Whitehill J, Bartlett M, Movellan J (2008) Automatic facial expression recognition for intelligent tutoring systems. In: IEEE computer society conference on computer vision and pattern recognition workshops, 2008. CVPRW’08, pp. 1–6. IEEE
Yang HF, Lin BY, Chang KY, Chen CS (2018) Joint estimation of age and expression by combining scattering and convolutional networks. ACM Trans Multimed Comput Commun Appl (TOMM) 14(1):1–18
Zhu Y, Li Y, Mu G, Guo G (2015) A study on apparent age estimation. In: proceedings of the IEEE international conference on computer vision workshops, pp. 25–31