Non-intrusive assessment of learners’ prior knowledge in dialogue-based intelligent tutoring systems
Tóm tắt
This article describes a study whose goal was to assess students’ prior knowledge level with respect to a target domain based solely on characteristics of the natural language interaction between students and conversational Intelligent Tutoring Systems (ITSs). We report results on data collected from two conversational ITSs: a micro-adaptive-only ITS and a fully-adaptive (micro- and macro-adaptive) ITS. These two ITSs are in fact different versions of the state-of-the-art conversational ITS DeepTutor (
http://www.deeptutor.org
). Our models rely on both dialogue and session interaction features including time on task, student generated content features (e.g., vocabulary size or domain specific concept use), and pedagogy-related features (e.g., level of scaffolding measured as number of hints). Linear regression models were explored based on these features in order to predict students’ knowledge level, as measured with a multiple-choice pre-test, and yielded in the best cases an r=0.949 and adjusted r-square =0.833. We discuss implications of our findings for the development of future ITSs.
Tài liệu tham khảo
BS Bloom, The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educ. Res. 13:, 4–16 (1984).
MTH Chi, SA Siler, H Jeong, Can tutors monitor students’ understanding accurately?Cogn. Instr. 22(3), 363–387 (2004).
MTH Chi, SA Siler, H Jeong, T Yamauchi, RG Hausmann, Learning from human tutoring. Cogn. Sci. 25(4), 471–533 (2001).
T Corcoran, FA Mosher, A Rogat, Learning progressions in science: An evidence-based approach to reform (CPRE Research Report #RR-63). Consortium for Policy Research in Education. University of Pennsylvania (2009). http://eric.ed.gov/?id=ED506730.
K Forbes-Riley, DJ Litman, in Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics. Modelling user satisfaction and student learning in a spoken dialogue tutoring system with generic, tutoring, and user affect parameters (Association for Computational LinguisticsNew York, New York, 2006), pp. 264–271.
AC Graesser, NK Person, JP Magliano, Collaborative dialogue patterns in naturalistic one-to-one tutoring. Appl. Cogn. Psychol. 9:, 495–522 (1995).
D Hestenes, M Wells, G Swackhamer, Force concept inventory. Phys. Teach.30(3), 141–158 (1992).
M Lintean, V Rus, R Azevedo, Automatic detection of student mental models during prior knowledge activation in metatutor. Int. J. Artif. Intell. Educ.21(3), 169–190 (2012).
L Mohan, J Chen, CW Anderson, Developing a multi-year learning progression for carbon cycling in socio-ecological systems. J. Res. Sci. Teach.46(6), 675–698 (2009).
C Moldovan, V Rus, AC Graesser, in The Proceedings of 22nd Midwest Artificial Intelligence and Cognitive Science Conference. Automated speech act classification for online chat, (2011), pp. 23–29.
P Pavlik, JR Anderson, Using a model to compute the optimal schedule of practice. J. Exp. Psychol. Appl.14(2), 101 (2008).
JW Pennebaker, ME Francis, RJ Booth, Linguistic inquiry and word count: Liwc 2001. Mahway: Lawrence Erlbaum Associates. 71:, 2001 (2001).
HL Roediger, JD Karpicke, The power of testing memory: Basic research and implications for educational practice. Perspect. Psychol. Sci.1(3), 181–210 (2006).
C Romero, M-I López, J-M Luna, S Ventura, Predicting students’ final performance from participation in on-line discussion forums. Comput. Educ. 68:, 458–472 (2013).
R Rus, W Baggett, E Gire, D Franceschetti, M Conley, A Graesser, in Design Recommendations for Intelligent Tutoring Systems: Learner Modeling, 1, ed. by R Sottilare, AC Graesser, X Hu, and H Holden. Toward learner models based on Learning Progressions in DeepTutor (Army Research LaboratoryOrlando, FL, 2013), pp. 183–192.
V Rus, S D’Mello, X Hu, A Graesser, Recent advances in conversational intelligent tutoring systems. AI Mag.34(3), 42–54 (2013).
V Rus, M Lintean, R Banjade, NB Niraula, D Stefanescu, in ACL (Conference System Demonstrations). Semilar: The semantic similarity toolkit (Citeseer, 2013), pp. 163–168.
V Rus, AC Graesser, W Baggett, D Franceschetti, D Stefanescu, N Niraula S Trausan-Matu, K Boyer, M Crosby, K Panou (eds.), Macro-adaptation in conversational intelligent tutoring matters. Automated response to questions with production rules (Springer International Publishing, Switzerland, 2014).
MD Shermis, J Burstein, Automated Essay Scoring: A Cross-disciplinary Perspective (Lawrence Erlbaum Associates, Inc., Hillsdale, NJ, 2003).
VJ Shute, Focus on formative feedback. Rev. Educ. Res. 78(1), 153–189 (2008).
VJ Shute, M Ventura, Stealth Assessment: Measuring and Supporting Learning in Video Games (MIT Press, Cambridge, 2013).
D Stefanescu, V Rus, AC Graesser, in Educational Data Mining 2014. Towards assessing students’ prior knowledge from tutorial dialogues (International Educational Data Mining Society, 2014).
R Taraban, K Rynearson, Computer-based comprehension research in a content area. J. Dev. Educ. 21(3), 10 (1998).
K VanLehn, The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educ. Psychol. 46(4), 197–221 (2011).
K VanLehn, AC Graesser, GT Jackson, PW Jordan, A Olney, CP Rosé, When are tutorial dialogues more effective than reading?Cogn. Sci. 31(1), 3–62 (2007).
LS Vygotsky, Mind in Society: The Development of Higher Psychological Processes (Harvard University Press, Cambridge, 1978).
C Williams, S D’Mello, in Intelligent Tutoring Systems. Predicting student knowledge level from domain-independent function and content words (SpringerSpringer-Verlag Berlin, Heidelberg, 2010), pp. 62–71.
BP Woolf, Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning (Morgan Kaufmann, Elsevier, Burlington, 2008).
J Yoo, J Kim, in Intelligent Tutoring Systems. Predicting learner’s project performance with dialogue features in online q&a discussions (Springer, 2012), pp. 570–575.