Engagement detection in online learning: a review

M. Al-Hendawi, Academic engagement of students with emotional and behavioral disorders: Existing research, issues, and future directions. Emotional and Behavioural Difficulties 17(2), 125–141 (2012)

T. Aluja-Baneta, M.-R. Sanchob, I. Vukic, Measuring motivation from the virtual learning environment in secondary education. Journal of Computational Science, 1–7 (2017)

A.R. Anderson, S.L. Christenson, M.F. Sinclair, C.A. Lehr, Check and connect: The importance of relationships for promoting engagement with school. J. Sch. Psychol. 42(2), 95–113 (2004)

S. Aslan, Z. Cataltepe, I. Diner, O. Dundar, A.A. Esme, R. Ferens, M. Yener, Learner Engagement Measurement and Classification in 1:1 Learning (International Conference on Machine Learning and Applications, Detroit, 2014)

M. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, J. Movellan, Automatic recognition of facial actions in spontaneous expressions. J. Multimed. 1(6), 22–35 (2006)

J.E. Beck, Engagement Tracing: Using Response Times Tomodel Student Disengagement (Conference on Artificial Intelligence in Education, Amsterdam, 2005)

B.M. Booth, A.M. Ali, S.S. Narayanan, I. Bennett, A.A. Farag, Toward Active and Unobtrusive Engagement Assessment of Distance Learners (International Conference on Affective Computing and Intelligent Interaction, San Antonio, 2017)

N. Bosch, Detecting Student Engagement: Human Versus Machine (Conference on User Modeling Adaptation and Personalization, Halifax, 2016)

N. Bosch, Y. Chen, S. D’Mello, It’s written on your face: Detecting affective states from facial expressions while learning computer programming (Intelligent Tutoring Systems, Honolulu, 2014)

N. Bosch, S.K. D’Mello, R.S. Baker, J. Ocumpaugh, V. Shute, M. Ventura, W. Zhao, Automatic Detection of Learning-Centered Affective States in the Wild (International Conference on Intelligent User Interfaces, Atlanta, 2015)

N. Bosch, S.K. D’Mello, R.S. Baker, J. Ocumpaugh, V. Shute, M. Ventura, W. Zhao, Detecting Student Emotions in Computer-Enabled Classrooms (International Joint Conference on Artificial Intelligence, New York, 2016)

G. Buscher, A. Dengel, L.v. Elst, Eye Movements as Implicit Relevance Feedback (Conference on Human Factors in Computing Systems, New York, 2008)

M. Chaouachi, P. Chalfoun, I. Jraidi, C. Frasson, Affect and Mental Engagement: Towards Adaptability for Intelligent Systems (Florida Artificial Intelligence Research Society Conference, Florida, 2010)

J. Chen, X. Liu, P. Tu, A. Aragones, Learning person-specific models for facial expression and action unit recognition. Pattern Recogn. Lett., 1964–1970 (2013)

S. Christenson, A. Reschly, C. Wylie, Handbook of Research on Student Engagement (Springer, New York, 2012)

S.L. Christenson, A.R. Anderson, The centrality of the learning context for students’ academic enabler skills. School Psychological Review 31(3), 378–393 (2002)

W.-S. Chu, F.D. Torre, J.F. Cohn, Selective transfer machine for personalized facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell., 529–545 (2017)

M. Cocea, S. Weibelzahl, Log file analysis for disengagement detection in e-learning environment. User Model. User-Adap. Inter. 19, 341–385 (2009)

M. Cocea, S. Weibelzahl, Disengagement detection in online learning: Validation studies and perspectives. IEEE Trans. Learn. Technol. 4(2), 114–124 (2011)

S. D’Mello, B. Lehman, R. Pekrun, A. Graesser, Confusion can be beneficial for learning. Learn. Instr. 29, 153–170 (2014)

S.K. D’Mello, S.D. Craig, A.C. Graesser, Multimethod assessment of affective experience and expression during deep learning. Int. J. Learn. Technol. 4(3), 165–187 (2009)

S.K. D’Mello, A. Graesser, Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Model. User-Adap. Inter. 20(2), 147–187 (2010)

A. Dewan, F. Lin, D. Wen, M. Murshed, Z. Uddin, A Deep Learning Approach to Detecting Engagement of Online Learners (IEEE International Conference on Internet of People, Guangzhou, 2018)

P. Ekman, W. Friesen, Facial Action Coding System: A Technique for the Measurement of Facial Movement (Consulting Psychologists Press, 1978)

P. Ekman, W. Friesen, J. Hager, Facial Action Coding System (A Human Face, 2002)

S. Fairclough, L. Venables, Prediction of subjective states from psychophysiology: A multivariate approach. Biol. Psychol. 71(1), 100–110 (2006)

J. Fei, I. Pavlidis, Thermistor at a distance: Unobtrusive measurement of breathing. IEEE Trans. Biomed. Eng. 5(4), 988–998 (2010)

M. Frank, G. Tofighi, H. Gu, R. Fruchter, Engagement Detection in Meetings (International Conference on Computing in Civil and Building Engineering, Osaka, 2016)

J.A. Fredrick, P.C. Blumenfeld, A.H. Paris, School engagement: Potential of the concept, state of the evidence. Rev. Educ. Res. 74(1), 59–109 (2004)

B. Goldberg, R. Sottilare, K. Brawner, H. Holden, Predicting Learner Engagement during Well-Defined and Ill-Defined Computer-Based Intercultural Interactions (International Conference on Affective Computing and Intelligent Interaction, Memphis, 2011)

J.F. Grafsgaard, J.B. Wiggins, K.E. Boyer, E.N. Wiebe, J.C. Lester, Automatically Recognizing Facial Indicators of Frustration: A Learning-Centric Analysis (International Conference on Affective Computing & Intelligent Interaction, Geneva, 2013a)

J.F. Grafsgaard, J.B. Wiggins, K.E. Boyer, E.N. Wiebe, J.C. Lester, Automatically Recognizing Facial Expression: Predicting Engagement and Frustration (International Conference on Educational Data Mining, Memphis, 2013b)

J.F. Grafsgaard, J.B. Wiggins, K.E. Boyer, E.N. Wiebe, J.C. Lester, Embodied affect in tutorial dialogue: Student gesture and posture (Artificial Intelligence in Education, Memphis, 2013c)

A. Gupta, A. D’Cunha, K. Awasthi, V. Balasubramanian, DAiSEE: Towards user engagement recognition in the wild. Computer Vision and Image Processing. arXiv preprint arXiv 1609, 01885 (2018)

S.L. Happy, A. Dasgupta, P. Patnaik, A. Routray, Automated Alertness and Emotion Detection for Empathic Feedback during e-Learning (IEEE 5th International Conference on Technology for Education, Kharagpur, 2013)

J. Harel, C. Koch, P. Perona, Graph-Based Visual Saliency (International Conference on Neural Information Processing Systems, Vancouver, 2006)

L.R. Harris, A phenomenographic investigation of teacher conceptions of student engagement in learning. Aust. Educ. Res. 35(1), 57–79 (2008)

K. Hwang, C. Yang, Fuzzy Fusion for Affective State Assessment in Distance Learning Based on Image Detection (International Conference on Audio, Language and Image Processing, Shanghai, 2008)

A.L. Jeni, F.J. Cohn, D.F. Torre, Facing Imbalanced Data Data-Recommendadtions for the Use of Performance Metrics (Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, 2013)

J. Johns, B. Woolf, A Dynamic Mixture Model to Detect Student Motivation and Proficiency (Proceedings of the Twenty-First National Conference on Artificial Intelligence, Boston, 2006)

A. Kamath, A. Biswas, V. Balasubramanian, A Crowdsourced Approach to Student Engagement Recognition in e-Learning Environments (IEEE Conference on Applications of Computer Vision, New York, 2016)

A. Kapoor, R.W. Picard, Multimodal Affect Recognition in Learning Environments (ACM international conference on Multimedia, New York, 2005)

S. Karumbaiah, B. Woolf, R. Lizarralde, I. Arroyo, D. Allessio, N. Wixon, Addressing Student Behavior and Affect with Empathy and Growth Mindset (International Conference on Educational Data Mining, Wuhan, 2017)

A. Kaur, A. Mustafa, L. Mehta, A. Dhall, Prediction and localization of student engagement in the wild. Computer Vision and Image Processing arXiv, 1804.00858 (2018)

J. Khelfallah, Slama, & Hadj, J. B, Facial expression recognition for intelligent tutoring systems in remote laboratories platform. Procedia Computer Science 73, 274–281 (2015)

H.C. Koydemir, A. Ozcan, Wearable and implantable sensors for biomedical applications. Annu. Rev. Anal. Chem. 11, 127–146 (2018)

L.B. Krithika, P.G.G. Lakshmi, Student emotion recognition system (SERS) for e-learning improvement based on learner concentration metric. Procedia Computer Science 85, 767–776 (2016)

G. Littlewort, J. Whitehill, T. Wu, I. Fasel, M. Frank, J. Movellan, M. Bartlett, The computer expression recognition toolbox (CERT) (IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, Santa Barbara, 2011)

B. Martinez, M.F. Valstar, B. Jiang, Automatic analysis of facial actions: A survey. IEEE Trans. Affect. Comput., 1–22 (2017)

S.J. Mason, A.P. Weigel, A generic forecast verification framework for administrative purposes. Mon. Weather Rev. 137(1), 331–349 (2009)

G. Matthews, S. Campbell, S. Falconer, L. Joyner, J. Huggins, K. Gilliland, J. Warm, Fundamental dimensions of subjective state in performance settings: Task engagement, distress, and worry. Emotion 2(4), 315–340 (2002)

B. McDaniel, S. D’Mello, B. King, P. Chipman, K. Tapp, A. Graesser, Facial Features for Affective State Detection in Learning Environments (Proceedings of the Annual Meeting of the Cognitive Science Society, California, 2007)

H. Monkaresi, N. Bosch, R. Calvo, S. D'Mello, Automated detection of engagement using video-based estimation of facial expressions and heart rate. IEEE Trans. Affect. Comput. 8(1), 15–28 (2017)

H.L. O'Brien, E.G. Toms, The development and evaluation of a survey to measure user engagement. J. Am. Soc. Inf. Sci. Technol. 61(1), 50–69 (2010)

J. Parsons, L. Taylor, Student engagement: what do we know and what should we do? (University of Alberta, Technical Report, Edmonton, 2011)

K. Porayska-Pomsta, M. Mavrikis, S. D'Mello, C. Conati, R. Baker, Knowledge elicitation methods for affect modelling in education. Int. J. Artif. Intell. Educ. 22(3), 107–140 (2013)

A. Psaltis, K.C. Apostolakis, K. Dimitropoulos, P. Daras, Multimodal student engagement recognition in prosocial games. IEEE Transactions on Computational Intelligence and AI in Games (2017)

S. Raina, L. Bernard, B. Taylor, S. Kaza, Using Eye-Tracking to Investigate Content Skipping: A Study on Learning Modules in Cybersecurity (IEEE Conference on Intelligence and Security Informatics, Tucson, 2016)

L. Rothkrantz, Dropout Rates of Regular Courses and MOOCs (International Conference on Computer Supported Education, Rome, 2016)

M. Saneiro, O.C. Santos, S. Salmeron-Majadas, J.G. Boticario, Towards emotion detection in educational scenarios from facial expressions and body movements through multimodal approaches. Sci. World J., 1–14 (2014)

S. Sathayanarayana, R.K. Satzoda, A. Carini, M. Lee, L. Salamanca, J. Reilly, G. Littlewort, Towards Automated Understanding of Student-Tutor Interactions Using Visual Deictic Gestures (IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, 2014)

M. Sathik, S.G. Jonathan, Effect of facial expressions on student’s comprehension recognition in virtual educational environments. SpringerPlus 2(455), 1–9 (2013)

D.J. Shernoff, M. Csikszentmihalyi, B. Schneider, E.S. Shernoff, Student engagement in high school classrooms from the perspective of flow theory. Sociol. Educ. 73, 247–269 (2000)

P. Sundar, S. Kumar, Disengagement detection in online learning using log file analysis. International journal of computer technology and applications 9(27), 195–301 (2016)

G. Tofighi, H. Gu, K. Raahemifar, Vision-based engagement detection in virtual reality (The Digital Media Industry and Academic Forum, Santorini, 2016)

A.K. Vail, J.F. Grafsgaard, K.E. Boyer, E.N. Wiebe, J.C. Lester, Predicting Learning from Student Affective Response to Tutor Questions (International Conference on Intelligent Tutoring Systems, Zagreb, 2016a)

A.K. Vail, J.B. Wiggins, J.F. Grafsgaard, K.E. Boyer, E.N. Wiebe, J.C. Lester, The Affective Impact of Tutor Questions: Predicting Frustration and Engagement (International Conference on Educational Data Mining, Raleigh, 2016b)

M.F. Valstar, E. Sanchez-Lozano, J.F. Cohn, A. Jeni, L. A, J.M. Girard, Z. Zhang, M. Pantic, Addressing head pose in the third facial expression recognition and analysis challenge. Computer Vision and Pattern Recognition (2017)

Z. Wang, Y. Li, S. Wang, Q. Li, Capturing Global Semantic Relationships for Facial Action Unit Recognition (IEEE International Conference on Computer Vision, Sydney, 2013)

J. Whitehill, M. Bartlett, J. Movellan, Automatic Faacial Expression Recognition for Intelligent Tutoring Systems (IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, 2008)

J. Whitehill, Z. Serpell, Y.-C. Lin, A. Foster, J. Movel, The faces of engagement: Automatic recognition of student engagement from facial expressions. IEEE Transactions on Affective Computing 5(1), 86–98 (2014)

N. Wixon, D. Allessio, S. Schultz , W. Burleson, I. Arroyo, K. Muldner, B.Woolf (2016). . Conference on User Modeling Adaptation and Personalization. Halifax