Multimodal data capabilities for learning: What can multimodal data tell us about learning?
Tóm tắt
Most research on learning technology uses clickstreams and questionnaires as their primary source of quantitative data. This study presents the outcomes of a systematic literature review of empirical evidence on the capabilities of multimodal data (MMD) for human learning. This paper provides an overview of what and how MMD have been used to inform learning and in what contexts. A search resulted in 42 papers that were included in the analysis. The results of the review depict the capabilities of MMD for learning and the ongoing advances and implications that emerge from the employment of MMD to capture and improve learning. In particular, we identified the six main objectives (ie, behavioral trajectories, learning outcome, learning‐task performance, teacher support, engagement and student feedback) that the MMLA research has been focusing on. We also summarize the implications derived from the reviewed articles and frame them within six thematic areas. Finally, this review stresses that future research should consider developing a framework that would enable MMD capacities to be aligned with the research and learning design (LD). These MMD capacities could also be utilized on furthering theory and practice. Our findings set a baseline to support the adoption and democratization of MMD within future learning technology research and development.
What is already known about this topic
Capturing and measuring learners’ engagement and behavior using MMD has been explored in recent years and exhibits great potential. There are documented challenges and opportunities associated with capturing, processing, analyzing and interpreting MMD to support human learning. MMD can provide insights into predicting learning engagement and performance as well as into supporting the process. What this paper adds
Provides a systematic literature review (SLR) of empirical evidence on MMD for human learning. Summarizes the insights MMD can give us about the learning outcomes and process. Identifies challenges and opportunities of MMD to support human learning. Implications for practice and/or policy
Learning analytics researchers will be able to use the SLR as a guide for future research. Learning analytics practitioners will be able to use the SLR as a summary of the current state of the field.
Từ khóa
Tài liệu tham khảo
Ahn T. B., 2020, Exploring emotions and multimodal learning analytics: Eye‐tracking and facial recognition, British Journal of Educational Technology
Baker R. S., 2015, The Oxford handbook of affective computing, 233
D’Mello S. K., 2005, Workshop at the International Conference on Intelligent User Interfaces, 7
Georgiadis K., 2015, International Conference on Games and Learning Alliance, 517
Giannakos M., 2020, Companion Proceedings of LAK20
Kidzinski Ł., 2016, Proceedings of the Ninth International Conference on Educational Data Mining, 406
Kitchenham B., 2007, Guidelines for performing systematic literature reviews in software engineering
Muñoz‐Cristóbal J. A., 2017, Joint Proceedings of the Sixth Multimodal Learning Analytics (MMLA) Workshop and the Second Cross‐LAK Workshop Co‐located with Seventh International Learning Analytics and Knowledge Conference, 60
Newell A., 1994, Unified Theories Of Cognition
Nicol D., 2013, Feedback in higher and professional education: understanding it and doing it well, 44
Paquette L., 2016, Sensor‐free or sensor‐full: A comparison of data modalities in multi‐channel affect detection
Sharma K., 2020, Assessing cognitive performance using physiological and facial features: Generalizing across contexts, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4, 10.1145/3411811
Sharma K., 2019, Thirteenth International Conference on Computer Supported Collaborative Learning (CSCL), 684
Vujovic M., 2020, Studying collaborative learning and space design with multimodal learning analytics, British Journal of Educational Technology
Worsley M., 2018, Multimodal learning analytics’ past, present, and potential futures