Children with autism spectrum disorder produce more ambiguous and less socially meaningful facial expressions: an experimental study using random forest classifiers
Tóm tắt
Computer vision combined with human annotation could offer a novel method for exploring facial expression (FE) dynamics in children with autism spectrum disorder (ASD). We recruited 157 children with typical development (TD) and 36 children with ASD in Paris and Nice to perform two experimental tasks to produce FEs with emotional valence. FEs were explored by judging ratings and by random forest (RF) classifiers. To do so, we located a set of 49 facial landmarks in the task videos, we generated a set of geometric and appearance features and we used RF classifiers to explore how children with ASD differed from TD children when producing FEs. Using multivariate models including other factors known to predict FEs (age, gender, intellectual quotient, emotion subtype, cultural background), ratings from expert raters showed that children with ASD had more difficulty producing FEs than TD children. In addition, when we explored how RF classifiers performed, we found that classification tasks, except for those for sadness, were highly accurate and that RF classifiers needed more facial landmarks to achieve the best classification for children with ASD. Confusion matrices showed that when RF classifiers were tested in children with ASD, anger was often confounded with happiness. The sample size of the group of children with ASD was lower than that of the group of TD children. By using several control calculations, we tried to compensate for this limitation. Children with ASD have more difficulty producing socially meaningful FEs. The computer vision methods we used to explore FE dynamics also highlight that the production of FEs in children with ASD carries more ambiguity.
Tài liệu tham khảo
Sullivan MW, Lewis M. Emotional expressions of young infants and children: a practitioner’s primer. Infants Young Children. 2003;16(2):120–42.
Oster H. The repertoire of infant facial expressions: an ontogenetic perspective. Emot Dev. 2005:261–92.
Sroufe LA. Emotional development. NewYork: Cambridge University Press; 1996.
Camras LA, Lambrecht L, Michel GF. Infant “surprise” expressions as coordinative motor structures. J Nonverbal Behav. 1996;20(3):183–95.
Ekman P, Roper G, Hager JC. Deliberate facial movement. Child Dev. 1980:886–91.
Grossard C, Chaby L, Hun S, et al. Children facial expression production: influence of age, gender, emotion subtype, elicitation condition and culture. Front Psychol. 2018;9:446.
Brun P. Psychopathologie de l’émotion chez l’enfant: l’importance des données développementales typiques. Enfance. 2001;53(3):281–91.
Field TM, Walden TA. Production and discrimination of facial expressions by preschool children. Child Develop. 1982:1299–311.
Komatsu S, Hakoda Y. Construction and evaluation of a facial expression database of children. Shinrigaku Kenkyu. 2012;83(3):217–24.
Camras LA, Oster H, Ujiie T, Campos JJ, Bakeman R, Meng Z. Do infants show distinct negative facial expressions for fear and anger? Emotional expression in 11-month-old European American, Chinese, and Japanese infants. Infancy. 2007;11(2):131–55.
Muzard A, Kwon AY, Espinosa N, Vallotton CD, Farkas C. Infants’ emotional expression: differences in the expression of pleasure and discomfort between infants from Chile and the United States. Infant Child Develop. 2017;26(6):e2033.
Sasson NJ, Faso DJ, Nugent J, Lovell S, Kennedy DP, Grossman RB. Neurotypical peers are less willing to interact with those with autism based on thin slice judgments. Sci Rep. 2017;7:40700.
Stagg SD, Slavny R, Hand C, Cardoso A, Smith P. Does facial expressivity count? How typically developing children respond initially to children with autism. Autism. 2014;18(6):704–11.
Trevisan DA, Hoskyn M, Birmingham E. Facial expression production in autism: a meta-analysis. Autism Res. 2018;11(12):1586–601.
Yirmiya N, Kasari C, Sigman M, Mundy P. Facial expressions of affect in autistic, mentally retarded and normal children. J Child Psychol Psychiatry. 1989;30(5):725–35.
Grossman RB, Edelson LR, Tager-Flusberg H. Emotional facial and vocal expressions during story retelling by children and adolescents with high-functioning autism. J Speech Lang Hear Res. 2013;56(3):1035–44. https://doi.org/10.1044/1092-4388(2012/12-0067).
Capps L, Kasari C, Yirmiya N, Sigman M. Parental perception of emotional expressiveness in children with autism. J Consult Clin Psychol. 1993;61(3):475.
Zane E, Neumeyer K, Mertens J, Chugg A, Grossman RB. I think we’re alone now: solitary social behaviors in adolescents with autism spectrum disorder. Journal of abnormal child psychology. 2018;46(5):1111–20.
Kasari C, Sigman M, Yirmiya N. Focused and social atten- tion of autistic children in interactions with familiar and unfamiliar adults: a comparison of autistic, mentally retarded, and normal children. Develop Psychopathol. 1993;5(3):403–14 https://doi.org/10.1017/S0954579400004491.
Loveland KA, Tunali-Kotoski B, Pearson DA, Brelsford KA, Ortegon J, Chen R. Imitation and expression of facial affect in autism. Develop Psychopathol. 1994;6(3):433–44.
Brewer R, Biotti F, Catmur C, Press C, Happé F, Cook R, Bird G. Can neurotypical individuals read autistic facial expressions? Atypical production of emotional facial expressions in autism spectrum disorders. Autism Res. 2016;9(2):262–71.
Volker MA, Lopata C, Smith DA, Thomeer ML. Facial encoding of children with high-functioning autism spectrum disorders. Focus Autism Other Develop Disabil. 2009;24(4):195–204.
Faso DJ, Sasson NJ, Pinkham AE. Evaluating posed and evoked facial expressions of emotion from adults with autism spectrum disorder. J Autism Dev Disord. 2015;45(1):75–89.
Senju A. Atypical development of spontaneous social cognition in autism spectrum disorders. Brain Dev. 2013;35(2):96–101.
Dapogny A, Bailly K, Dubuisson S. Pairwise conditional random forests for facial expression recognition. In: Proceedings of the IEEE international conference on computer vision; 2015. p. 3783–91.
Samad MD. Computational modeling of facial response for detecting differential traits in Autism Spectrum Disorders [Doctor of Philosophy (PhD), dissertation, Electrical/Computer Engineering] Old Dominion University. doi: https://doi.org/10.25777/q96j-7v27
Metallinou A, Grossman RB, Narayanan S. Quantifying atypicality in affective facial expressions of children with autism spectrum disorders. Proc IEEE Int Conf Multimed Expo. 2013:1–6. https://doi.org/10.1109/ICME.2013.6607640.
Guha T, Yang Z, Grossman RB, Narayanan SS. A computational study of expressive facial dynamics in children with autism. IEEE Trans Affect Comput. 2018;9(1):14–20. https://doi.org/10.1109/TAFFC.2016.2578316.
Zane E, Yang Z, Pozzan L, Guha T, Narayanan S, Grossman RB. Motion-capture patterns of voluntarily mimicked dynamic facial expressions in children and adolescents with and without ASD. J Autism Dev Disord. 2019;49(3):1062–79. https://doi.org/10.1007/s10803-018-3811-7.
Owada K, Kojima M, Yassin W, et al. Computer-analyzed facial expression as a surrogate marker for autism spectrum social core symptoms. PloS One. 2018;13(1):e0190442.
Leo M, Carcagnì P, Distante C, et al. Computational assessment of facial expression production in ASD children. Sensors. 2018;18(11):3993.
Martinez B, Valstar MF, Jiang B, Pantic M. Automatic analysis of facial actions: a survey. IEEE Transac Affect Comput. 2019;10(3):325–47, 1. https://doi.org/10.1109/TAFFC.2017.2731763.
Klin A, Jones W, Schultz R, Volkmar F, Cohen D. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Arch Gen Psychiatry. 2002;59(9):809–16.
Belin P, Fillion-Bilodeau S, Gosselin F. The Montreal Affective Voices: a validated set of nonverbal affect bursts for research on auditory affective processing. Behav Res Methods. 2008;40(2):31–539.
Egger HL, Pine DS, Nelson E, Leibenluft E, Ernst M, Towbin KE, Angold A. The NIMH Child Emotional Faces Picture Set (NIMH-ChEFS): a new set of children’s facial emotion stimuli. Int J Methods Psychiatr Res. 2011;20(3):145–56.
Mazurski EJ, Bond NW. A new series of slides depicting facial expressions of affect: a comparison with the pictures of facial affect series. Aust J Psychol. 1993;45(1):41–7.
Bänziger T, Mortillaro M, Scherer KR. Introducing the Geneva Multimodal Expression corpus for experimental research on emotion perception. Emotion. 2012;12(5):1161–79.
Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. CVPR. 2001;1(511-518):3.
Xiong X, De la Torre F. Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2013. p. 532–9.
Dalal N, Triggs B. Histograms of oriented gradients for human detection; 2005.
Breiman L. Random forests. Machine Learning. 2001;45(1):5–32.
Camras LA, Chen Y, Bakeman R, Norris K, Cain TR. Culture, ethnicity, and children’s facial expressions: a study of European American, mainland Chinese, Chinese American, and adopted Chinese girls. Emotion. 2006;6(1):103–14.
Vivanti G, Hamilton A. Imitation in autism spectrum disorders. In: Volkmar FR, Paul R, Rogers SJ, Pelphrey KA, editors. Handbook of autism and pervasive developmental disorders, fourth edition: Wiley; 2014.
Dapogny A, Bailly K, Dubuisson S. Confidence-weighted local expression predictions for occlusion handling in expression recognition and action unit detection. Int J Comp Vision. 2018;126(2–4):255–71.
Riggio HR, Riggio RE. Extraversion, neuroticism, and emotional expressiveness: a meta-analysis. J Nonvl Behav. 2002;26:195–218.