Face-Based Attention Recognition Model for Children with Autism Spectrum Disorder

James W (1890) The principles of psychology New York. Holt and company

Whitehill J, Serpell Z, Lin Y-C, Foster A, Movellan JR (2014) The faces of engagement: automatic recognition of student engagementfrom facial expressions. IEEE Transactions on Affective Computing 5:86–98

Henrie CR, Halverson LR, Graham CR (2015) Measuring student engagement in technology-mediated learning: a review. Computers & Education 90:36–53. https://doi.org/10.1016/j.compedu.2015.09.005

Association AP (2015) Guidelines for psychological practice with transgender and gender nonconforming people. 70:832–864

Maenner MJ, Shaw KA, Baio J (2020) Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2016. MMWR Surveillance Summaries 69:1

Church BA, Rice CL, Dovgopoly A, Lopata CJ, Thomeer ML, Nelson A, Mercado E 3rd. (2015) Learning, plasticity, and atypical generalization in children with autism. Psychonomic Bulletin & Review 22:1342–1348. https://doi.org/10.3758/s13423-014-0797-9

Almendros MLR, Cuevas MC, Domínguez CR, López TR, Bermúdez-Edo M, Fórtiz MJR (2016) A tool to improve visual attention and the acquisition of meaning for low-functioning people. In Proceedings of International Conference on Computers Helping People with Special Needs:234–241

Alcorn A, Pain H, Rajendran G, Smith T, Lemon O, Porayska-Pomsta K, Foster ME, Avramides K, Frauenberger C, Bernardini S (2011) Social communication between virtual characters and children with autism. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 6738:7–14

Kurniawan I (2018) The improvement of autism spectrum disorders on children communication ability with PECS method Multimedia Augmented Reality-Based. In Proceedings of Journal of Physics: Conference Series:012009

Dinesh D, Bijlani K (2016) Student analytics for productive teaching/learning. In Proceedings of International Conference on Information Science 97-102

Zaletelj J, Košir A (2017) Predicting students’ attention in the classroom from Kinect facial and body features. EURASIP journal on image and video processing 2017:80

Dewan MAA, Murshed M, Lin F (2019) Engagement detection in online learning: a review. Smart Learning Environments 6:1

Chen, Wang JY, Yu CM (2017) Assessing the attention levels of students by using a novel attention aware system based on brainwave signals. British Journal of Educational Technology 48:348–369

Simone Di P, Tonacci A, Narzisi A, Domenici C, Pioggia G, Muratori F, Billeci L (2017) Monitoring of autonomic response to sociocognitive tasks during treatment in children with autism spectrum disorders by wearable technologies: a feasibility study. Computers in Biology and Medicine 85:143–152. https://doi.org/10.1016/j.compbiomed.2016.04.001

Billeci L, Narzisi A, Tonacci A, Sbriscia-Fioretti B, Serasini L, Fulceri F, Apicella F, Sicca F, Calderoni S, Muratori F (2017) An integrated EEG and eye-tracking approach for the study of responding and initiating joint attention in autism spectrum disorders. Scientific Reports 7:13560

Dehzangi O, Williams C (2015) Towards multi-modal wearable driver monitoring: impact of road condition on driver distraction. In Proceedings of 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN). 1-6.

Mounia L, Heather OB, Elad Y-T (2014) Measuring user engagement. Morgan & Claypool:132. https://doi.org/10.2200/S00605ED1V01Y201410ICR038

Davis J, McKone E, Zirnsak M, Moore T, O’Kearney R, Apthorp D, Palermo R (2017) Social and attention-to-detail subclusters of autistic traits differentially predict looking at eyes and face identity recognition ability. British Journal of Psychology 108:191–219. https://doi.org/10.1111/bjop.12188

Mythili MS, Mohamed Shanavas AR (2016) Early prediction of cognitive disorders among children using Bee Hive optimization approach. (CODEO). Biomedical and Pharmacology Journal 9:615–621. https://doi.org/10.13005/bpj/981

Rinehart NJ, Bradshaw JL, Moss SA, Brereton AV, Tonge BJ (2008) Brief report: Inhibition of return in young people with autism and Asperger’s disorder. Autism 12:249–260. https://doi.org/10.1177/1362361307088754

Roy SD, Bhowmik MK, Saha P, Ghosh AK (2016) An approach for automatic pain detection through facial expression. Procedia Computer Science 84:99–106

Vezzetti E, Speranza D, Marcolin F, Fracastoro G, Buscicchio G (2014) Exploiting 3d ultrasound for fetal diagnostic purpose through facial landmarking. Image Analysis & Stereology 33:167–188

Jabbar R, Al-Khalifa K, Kharbeche M, Alhajyaseen W, Jafari M, Jiang S (2018) Real-time driver drowsiness detection for android application using deep neural networks techniques. Procedia computer science 130:400–407

Chu H-C, Tsai WW-J, Liao M-J, Chen Y-M (2018) Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learning. Soft Computing 22:2973–2999

Monkaresi H, Bosch N, Calvo RA, D’Mello SK (2016) Automated detection of engagement using video-based estimation of facial expressions and heart rate. IEEE Transactions on Affective Computing 8:15–28

Ko BC (2018) A brief review of facial emotion recognition based on visual information. Sensors 18:401

Sugiura M (2015) Three faces of self-face recognition: potential for a multi-dimensional diagnostic tool. Neuroscience Research 90:56–64. https://doi.org/10.1016/j.neures.2014.10.002

Zeng Z, Pantic M, Roisman GI, Huang TS (2008) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31:39–58

Ghimire D, Lee J (2013) Geometric feature-based facial expression recognition in image sequences using multi-class adaboost and support vector machines. Sensors 13:7714–7734

Sariyanidi E, Gunes H, Cavallaro A (2014) Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE transactions on pattern analysis and machine intelligence 37:1113–1133

Ghimire D, Jeong S, Lee J, Park SH (2017) Facial expression recognition based on local region specific features and support vector machines. Multimedia Tools and Applications 76:7803–7821

Poursaberi A, Noubari HA, Gavrilova M, Yanushkevich SN (2012) Gauss–Laguerre wavelet textural feature fusion with geometrical information for facial expression identification. EURASIP Journal on Image and Video Processing 2012:17

Kotsia I, Pitas I (2006) Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE transactions on image processing 16:172–187

Rudovic, O.; Pavlovic, V.; Pantic, M. (2012) Multi-output Laplacian dynamic ordinal regression for facial expression recognition and intensity estimation. In Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition, 16-21. 2634-2641.

Sung J, Kim D (2009) Real-time facial expression recognition using STAAM and layered GDA classifier. Image and Vision Computing 27:1313–1325

Majumder A, Behera L, Subramanian VK (2014) Emotion recognition from geometric facial features using self-organizing map. Pattern Recognition 47:1282–1293

Soyel, H., Demirel, H (2007) Facial expression recognition using 3D facial feature distances. In Proceedings of International Conference Image Analysis and Recognition 831-838.

Li, X.; Ruan, Q.; Ming, Y. (2010) 3D facial expression recognition based on basic geometric features. In Proceedings of IEEE 10th Internatonal Conference on Signal Processing 1366-1369

Tang, H.; Huang, T.S. (2008) 3D facial expression recognition based on properties of line segments connecting facial feature points. In Proceedings of 8th IEEE International Conference on Automatic Face & Gesture Recognition. 1-6.

Soyel H, Demirel H (2010) Optimal feature selection for 3D facial expression recognition using coarse-to-fine classification. Turkish Journal of Electrical Engineering & Computer Sciences 18:1031–1040

Shan, K.; Guo, J.; You, W.; Lu, D.; Bie, R. (2017) Automatic facial expression recognition based on a deep convolutional-neural-network structure. In Proceedings of IEEE 15th International Conference on Software Engineering Research, Management and Applications (SERA). 123-128.

Bezawada S, Hu Q, Gray A, Brick T, Tucker C (2017) Automatic facial feature extraction for predicting designers’ comfort with engineering equipment during prototype creation. Journal of Mechanical Design 139

Chen, X.; Yang, X.; Wang, M.; Zou, J. (2017) Convolution neural network for automatic facial expression recognition. In Proceedings of International conference on applied system innovation (ICASI) 814-817

Nonis F, Dagnes N, Marcolin F, Vezzetti E (2019) 3D approaches and challenges in facial expression recognition algorithms—a literature review. Applied Sciences 9:3904

Hua W, Dai F, Huang L, Xiong J, Gui G (2019) HERO: human emotions recognition for realizing intelligent Internet of Things. IEEE Access 7:24321–24332

Wu B-F, Lin C-H (2018) Adaptive feature mapping for customizing deep learning based facial expression recognition model. IEEE access 6:12451–12461

Ng, H.-W.; Nguyen, V.D.; Vonikakis, V. (2015) Winkler, S. Deep learning for emotion recognition on small datasets using transfer learning. In Proceedings of Proceedings of the ACM on international conference on multimodal interaction. 443-449.

Levi, G.; Hassner, T. (2015) Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In Proceedings of Proceedings of the ACM on international conference on multimodal interaction. 503-510

Canedo D, Neves AJ (2019) Facial expression recognition using computer vision: a systematic review. Applied Sciences 9:4678

Azulay, A.; Weiss, Y (2018) Why do deep convolutional networks generalize so poorly to small image transformations? arXiv preprint arXiv:1805.12177

Association AP (2019) American psychiatric association

Williams J, Scott F, Stott C, Allison C, Bolton P, Baron-Cohen S, Brayne C (2005) The CAST (childhood Asperger syndrome test) test accuracy. Autism 9:45–68

iMotions iMotion Biometric Tool (2017)

Viola P, Jones, M (2001) Rapid object detection using a boosted cascade of simple features. In Proceedings of Proceedings of the IEEE computer society conference on computer vision and pattern recognition. CVPR I-I

McDuff, D.; Mahmoud, A.; Mavadati, M.; Amr, M.; Turcot, J.; Kaliouby, R.E (2016) AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. In Proceedings of Proceedings of the CHI conference extended abstracts on human factors in computing systems. 3723-3726

Senechal T, McDuff D, Kaliouby R (2015) Facial action unit detection using active learning and an efficient non-linear kernel approximation. In Proceedings of Proceedings of the IEEE International Conference on Computer Vision Workshops 10-18.

Kory Westlund JM, Jeong S, Park HW, Ronfard S, Adhikari A, Harris PL, DeSteno D, Breazeal CL (2017) Flat vs. expressive storytelling: young children’s learning and retention of a social robot’s narrative. Frontiers in human neuroscience 11:295

Magdin M, Prikler F (2018) Real time facial expression recognition using webcam and SDK affectiva. IJIMAI 5:7–15

Abdic I, Fridman L, McDuff D, Marchi E, Reimer B, Schuller B (2016) Driver frustration detection from audio and video in the wild. Proceedings of the KI 237

Sawyer, R.; Smith, A.; Rowe, J.; Azevedo, R.; Lester, J. (2017) Enhancing student models in game-based learning with facial expression recognition. In Proceedings of Proceedings of the 25th conference on user modeling, adaptation and personalization 192-201.

Huang, K.-C.; Huang, S.-Y.; Kuo, Y.-H. (2010) Emotion recognition based on a novel triangular facial feature extraction method. In Proceedings of The International Joint Conference on Neural Networks (IJCNN) 1-6

Steger, A.; Timofte, R. Failure detection for facial landmark detectors. In Proceedings of Asian Conference on Computer Vision 361-376.

Al Haj, M.; Gonzalez, J.; Davis, L.S. (2012) On partial least squares in head pose estimation: how to simultaneously deal with misalignment. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 2602-2609

Kendrick C, Tan K, Walker K, Yap MH (2018) Towards real-time facial landmark detection in depth data using auxiliary information. Symmetry 10:230

Schölkopf B, Burges C, Vapnik, V (1996) Incorporating invariances in support vector learning machines. In Proceedings of International Conference on Artificial Neural Networks 47-52

Bosch N, D’mello SK, Ocumpaugh J, Baker RS, Shute V (2016) Using video to automatically detect learner affect in computer-enabled classrooms. ACM Transactions on Interactive Intelligent Systems (TiiS) 6:17

Kelkar BA, Rodd SF, Kulkarni UP (2019) Estimating distance threshold for greedy subspace clustering. Expert Systems with Applications 135:219–236

Martinez B, Valstar MF, Jiang B, Pantic M (2017) Automatic analysis of facial actions: a survey. IEEE transactions on affective computing

Valstar, M.F.; Gunes, H.; Pantic, M. How to distinguish posed from spontaneous smiles using geometric features. In Proceedings of Proceedings of the 9th international conference on Multimodal interfaces 38-45

Sajjad M, Zahir S, Ullah A, Akhtar Z, Muhammad K (2019) Human behavior understanding in big multimedia data using CNN based facial expression recognition. Mobile networks and applications:1–11

Li J, Jin K, Zhou D, Kubota N, Ju Z (2020) Attention mechanism-based CNN for facial expression recognition. Neurocomputing 411:340–350

Hudac CM, Stessman HA, DesChamps TD, Kresse A, Faja S, Neuhaus E, Webb SJ, Eichler EE, Bernier RA (2017) Exploring the heterogeneity of neural social indices for genetically distinct etiologies of autism. Journal of neurodevelopmental disorders 9:1–13

Hanley M, Khairat M, Taylor K, Wilson R, Cole-Fletcher R, Riby DM (2017) Classroom displays-attraction or distraction? Evidence of impact on attention and learning from children with and without autism. Developmental Psychology 53:1265–1275. https://doi.org/10.1037/dev0000271

Das, T.R.; Hasan, S.; Sarwar, S.; Das, J.K.; Rahman, M.A. Facial spoof detection using support vector machine. In Proceedings of Proceedings of International Conference on Trends in Computational and Cognitive Engineering 615-625

Putnam PT, Roman JM, Zimmerman PE, Gothard KM (2016) Oxytocin enhances gaze-following responses to videos of natural social behavior in adult male rhesus monkeys. Psychoneuroendocrinology 72:47–53. https://doi.org/10.1016/j.psyneuen.2016.05.016

Tovo-Rodrigues L, Recamonde-Mendoza M, Paixão-Côrtes VR, Bruxel EM, Schuch JB, Friedrich DC, Rohde LA, Hutz MH (2016) The role of protein intrinsic disorder in major psychiatric disorders. American Journal of Medical Genetics, Part B: Neuropsychiatric Genetics 171:848–860. https://doi.org/10.1002/ajmg.b.32455

Smith T, Iadarola S (2015) Evidence base update for autism spectrum disorder. Journal of Clinical Child and Adolescent Psychology 44:897–922. https://doi.org/10.1080/15374416.2015.1077448

Bayliss AP, di Pellegrino G, Tipper SP (2005) Sex differences in eye gaze and symbolic cueing of attention. The Quarterly journal of experimental psychology. A. Human experimental psychology 58:631–650

Bayliss AP, Tipper SP (2006) Predictive gaze cues and personality judgments: should eye trust you? Psychol Sci 17:514–520. https://doi.org/10.1111/j.1467-9280.2006.01737.x