Applying multimodal learning analytics to examine the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming

Barron, B. (2003). When smart groups fail. Journal of the Learning Sciences, 12(3), 307–359. https://doi.org/10.1207/S15327809JLS1203_1 Belland, B. R., Walker, A. E., Kim, N. J., & Lefler, M. (2017). Synthesizing results from empirical research on computer-based scaffolding in STEM education: A meta-analysis. Review of Educational Research, 87(2), 309–344. https://doi.org/10.3102/0034654316670999 Bezemer, J. (2008). Displaying orientation in the classroom: Students’ multimodal responses to teacher instructions. Linguistics and Education, 19(2), 166–178. https://doi.org/10.1016/j.linged.2008.05.005 Bliss, J., Askew, M., & MacRae, S. (1996). Effective teaching and learning: Scaffolding revisited. Oxford Review of Education, 22(1), 37–61. https://doi.org/10.1080/0305498960220103 Bloome, D., Beierle, M., Grigorenko, M., & Goldman, S. (2009). Learning over time: Uses of intercontextuality, collective memories, and classroom chronotopes in the construction of learning opportunities in a ninth-grade language arts classroom. Language and Education, 23(4), 313–334. https://doi.org/10.1080/09500780902954257 Brown, R., & Renshaw, P. (2009). Positioning students as actors and authors: A chronotopic analysis of collaborative learning activities. Mind, Culture and Activity, 13(3), 247–259. https://doi.org/10.1207/s15327884mca1303_6 Bulu, S. T., & Pedersen, S. (2010). Scaffolding middle school students’ content knowledge and ill-structured problem solving in a problem-based hypermedia learning environment. Educational Technology Research & Development, 58(5), 507–529. https://doi.org/10.1007/s11423-010-9150-9 Chatti, M. A., Muslim, A., & Schroeder, U. (2017). Toward an open learning analytics ecosystem. In B. K. Daniel (Ed.), Big data and learning analytics in higher education (pp. 195–219). Cham: Springer. https://doi.org/10.1007/978-3-319-06520-5_12 Chevalier, M., Giang, C., Piatti, A., & Mondada, F. (2020). Fostering computational thinking through educational robotics: A model for creative computational problem solving. International Journal of STEM Education, 7(1), 1–18. https://doi.org/10.1186/s40594-020-00238-z Chinese Ministry of Education. (2019). Report on the development of online learning in China 2019. Tsinghua University Press. Chiu, T. K. F. (2021). A holistic approach to the design of artificial intelligence (AI) education for K-12 schools. TechTrends, 65(5), 796–807. https://doi.org/10.1007/s11528-021-00637-1 Clark, D. B., & Sengupta, P. (2020). Reconceptualizing games for integrating computational thinking and science as practice: Collaborative agent-based disciplinarily-integrated games. Interactive Learning Environments, 28(3), 328–346. https://doi.org/10.1080/10494820.2019.1636071 Cress, U., Rosé, C., Wise, A. F., & Oshima, J. (2021). International handbook of computer-supported collaborative learning. Springer. https://doi.org/10.1007/978-3-030-65291-3 Damşa, C. I., & Nerland, M. (2016). Student learning through participation in inquiry activities: Two case studies in teacher and computer engineering education. Vocations and Learning, 9(3), 275–294. https://doi.org/10.1007/s12186-016-9152-9 Demir, O., & Seferoglu, S. S. (2020a). A comparison of solo and pair programming in terms of flow experience, coding quality, and coding achievement. Journal of Educational Computing Research, 58(8), 1448–1466. https://doi.org/10.1177/0735633120949788 Demir, O., & Seferoglu, S. S. (2020b). The effect of determining pair programming groups according to various individual difference variables on group compatibility, flow, and coding performance. Journal of Educational Computing Research, 59(1), 41–70. https://doi.org/10.1177/0735633120949787 Di Mitri, D., Schneider, J., & Drachsler, H. (2021). Keep me in the loop: Real-time feedback with multimodal data. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-021-00281-z Dillenbourg, P. (1999). What do you mean by “collaborative learning”? In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches (Vol. 1, pp. 1–15). Elsevier. Fanchamps, L. J. A., Slangen, L., Hennissen, P., & Specht, M. M. (2021). The influence of SRA programming on algorithmic thinking and self-efficacy using Lego robotics in two types of instruction. International Journal of Technology and Design Education, 31(2), 203–222. https://doi.org/10.1007/s10798-019-09559-9 Fedorenko, E., Ivanova, A., Dhamala, R., & Bers, M. U. (2019). The language of programming: A cognitive perspective. Trends in Cognitive Sciences, 23(7), 525–528. https://doi.org/10.1016/j.tics.2019.04.010 Gaul, C., & Kim, M. K. (2020). Learner participation regulation supported by long-term peer moderation and participation feedback during asynchronous discussions. Journal of Computers in Education, 7(3), 295–331. https://doi.org/10.1007/s40692-020-00158-5 Gidalevich, S., & Kramarski, B. (2019). The value of fixed versus faded self-regulatory scaffolds on fourth graders’ mathematical problem solving. Instructional Science, 47(1), 39–68. https://doi.org/10.1007/s11251-018-9475-z Goodyear, P., Jones, C., & Thomson, K. (2014). Computer-supported collaborative learning: Instructional approaches, group processes and educational designs. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 439–451). Springer. Gresalfi, M., Martin, T., Hand, V., & Greeno, J. (2009). Constructing competence: An analysis of student participation in the activity systems of mathematics classrooms. Educational Studies in Mathematics, 70(1), 49–70. https://doi.org/10.1515/ctra-2016-0019 Grévisse, C., Rothkugel, S., & Reuter, R. A. P. (2019). Scaffolding support through integration of learning material. Smart Learning Environments, 6(1), 1–24. https://doi.org/10.1186/s40561-019-0107-0 Grossen, M., & Bachmann, K. (2000). Learning to collaborate in a peer-tutoring situation: Who learns? What is learned? European Journal of Psychology of Education, 15(4), 491–508. https://doi.org/10.1007/BF03172990 Guzdial, M., & Morrison, B. (2016). Growing computer science education into a STEM education discipline. Communications of the ACM, 59(11), 31–33. https://doi.org/10.1145/3000612 Holmes, N. G., Day, J., Park, A. H. K., Bonn, D. A., & Roll, I. (2014). Making the failure more productive: Scaffolding the invention process to improve inquiry behaviors and outcomes in invention activities. Instructional Science, 42(4), 523–538. https://doi.org/10.1007/s11251-013-9300-7 Hmelo-Silver, C. E., & DeSimone, C. (2013). Problem-based learning: An instructional model of collaborative learning. In C. E. Hmelo-Silver, C. A. Chinn, C. K. K. Chan, & A. M. O’Donnell (Eds.), The international handbook of collaborative learning (1st ed., pp. 370–385). New York, NY: Routledge. Hwang, W. Y., Shadiev, R., Wang, C. Y., & Huang, Z. H. (2012). A pilot study of cooperative programming learning behavior and its relationship with students’ learning performance. Computers & Education, 58(4), 1267–1281. https://doi.org/10.1016/j.compedu.2011.12.009 Jadallah, M., Anderson, R. C., Nguyen-Jahiel, K., Miller, B. W., Kim, I. H., Kuo, L. J., Dong, T., & Wu, X. (2011). Influence of a teacher’s scaffolding moves during child-led small group discussions. American Educational Research Journal, 48(1), 194–230. https://doi.org/10.3102/0002831210371498 Johnson, E. K. (2019). Waves: Scaffolding self-regulated learning to teach science in a whole-body educational game. Journal of Science Education and Technology, 28(2), 133–151. https://doi.org/10.1007/s10956-018-9753-1 Khan, S. M. (2017). Multimodal behavioral analytics in intelligent learning and assessment systems. Innovative assessment of collaboration (pp. 173–184). Springer. https://doi.org/10.1007/978-3-319-33261-1_11 Kiemer, K., Gröschner, A., Pehmer, A. K., & Seidel, T. (2015). Effects of a classroom discourse intervention on teachers’ practice and students’ motivation to learn mathematics and science. Learning and Instruction, 35, 94–103. https://doi.org/10.1016/j.learninstruc.2014.10.003 Kim, N. J., Vicentini, C. R., & Belland, B. R. (2022). Influence of scaffolding on information literacy and argumentation skills in virtual field trips and problem-based learning for scientific problem solving. International Journal of Science and Mathematics Education, 20(2), 215–236. https://doi.org/10.1007/s10763-020-10145-y Kirschner, P., Sweller, J., & Clark, R. (2006). Why unguided learning does not work: An analysis of the failure of discovery learning, problem-based learning, experiential learning and inquiry-based learning. Educational Psychologist, 41(2), 75–86. https://doi.org/10.1207/s15326985ep4102_1. Kunkle, W. M., & Allen, R. B. (2016). The impact of different teaching approaches and languages on student learning of introductory programming concepts. ACM Transactions on Computing Education, 16(1), 1–26. https://doi.org/10.1145/2785807 Kwon, K., Hong, R., & Laffey, J. M. (2013). The educational impact of metacognitive group coordination in computer-supported collaborative learning. Computers in Human Behavior, 29(4), 1271–1281. https://doi.org/10.1016/j.chb.2013.01.003 Kynigos, C., & Diamantidis, D. (2022). Creativity in engineering mathematical models through programming. ZDM-Mathematics Education, 54, 149–162. https://doi.org/10.1007/s11858-021-01314-6 Lai, X., & Wong, G. K. W. (2022). Collaborative versus individual problem solving in computational thinking through programming: A meta-analysis. British Journal of Educational Technology, 53(1), 150–170. https://doi.org/10.1111/bjet.13157 Lavonen, J., Meisalo, V., & Lattu, M. (2002). Collaborative problem solving in a control technology learning environment, a pilot study. International Journal of Technology and Design Education, 12(2), 139–160. https://doi.org/10.1023/A:1015261004362 Lewis, C. (2012). The importance of students’ attention to program state: A case study of debugging behavior. In C. Alison, S. Kate, & S. Beth (Eds.), Proceedings of the 9th annual international conference on international computing education research (pp. 127–134). Association for Computing Machinery. Lin, Y., Wu, C., & Chiu, C. (2018). The use of wiki in teaching programming: Effects upon achievement, attitudes, and collaborative programming behaviors. International Journal of Distance Education Technologies, 16(3), 18–45. https://doi.org/10.4018/IJDET.2018070102 Maksic, S., & Josic, S. (2021). Scaffolding the development of creativity from the students’ perspective. Thinking Skills and Creativity, 41, 100835. https://doi.org/10.1016/j.tsc.2021.100835 Margulieux, L. E., & Catrambone, R. (2021). Scaffolding problem solving with learners’ own self explanations of subgoals. Journal of Computing in Higher Education, 33(2), 499–523. https://doi.org/10.1007/s12528-021-09275-1 Mattanah, J. F., Pratt, W. P., Cowan, P. A., & Cowan, C. P. (2005). Authoritative parenting, parental scaffolding of long-division mathematics, and children’s academic competence in fourth grade. Journal of Applied Developmental Psychology, 26(1), 85–106. https://doi.org/10.1016/j.appdev.2004.10.007. Meloth, M. S., & Deering, P. D. (1999). The role of the teacher in promoting cognitive processing during collaborative learning. In A. M. O’Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 235–255). Lawrence Erlbaum Associates Publishers. Nedić, J., Jošić, S., & Baucal, A. (2015). The role of asymmetrical interaction in the assessment of nonverbal abilities of children from the drop-in center. Teaching Innovations, 28(3), 189–206. https://doi.org/10.5937/inovacije1503189N Ouyang, F., Chang, Y. H., Scharber, C., Jiao, P., & Huang, T. (2020). Examining the instructor-student collaborative partnership in an online learning community course. Instructional Science, 48(2), 183–204. https://doi.org/10.1007/s11251-020-09507-4 Ouyang, F., Chen, S., Yang, Y., & Chen, Y. (2022). Examining the effects of three group-level metacognitive scaffoldings on in-service teachers’ knowledge building. Journal of Educational Computing Research, 60(2), 352–379. https://doi.org/10.1177/07356331211030847 Ouyang, F., Chen, Z., Cheng, M., Tang, Z., & Su, C.-Y. (2021). Exploring the effect of three scaffoldings on the collaborative problem-solving processes in China’s higher education. International Journal of Educational Technology in Higher Education, 18(35), 1–22. https://doi.org/10.1186/s41239-021-00273-y Ouyang, F., & Xu, W. (2022). The effects of three instructor participatory roles on a small group’s collaborative concept mapping. Journal of Educational Computing Research, 60(4), 930–959. https://doi.org/10.1177/07356331211057283 Papadakis, S. (2018). Is pair programming more effective than solo programming for secondary education novice programmers? A case study. International Journal of Web-Based Learning and Teaching Technologies, 13(1), 1–16. https://doi.org/10.1177/1555412015622345 Pedró, F., Subosa, M., Rivas, A., & Valverde, P. (2019). Artificial intelligence in education: Challenges and opportunities for sustainable development. United Nations Educational, Scientific and Cultural Organization. Perret-Clermont, A. N. (1980). Social interaction and cognitive development in children. Academic Press. Plonka, L., Sharp, H., Van der Linden, J., & Dittrich, Y. (2015). Knowledge transfer in pair programming: An in-depth analysis. International Journal of Human–computer Studies, 73, 66–78. https://doi.org/10.1016/j.ijhcs.2014.09.001 Rasku-Puttonen, H., Eteläpelto, A., Arvaja, M., & Häkkinen, P. (2003). Is successful scaffolding an illusion?—Shifting patterns of responsibility and control in teacher–student interaction during a long-term learning project. Instructional Science, 31(6), 377–393. https://doi.org/10.1023/A:1025700810376 Riikonen, S., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2020). Bringing maker practices to school: Tracing discursive and materially mediated aspects of student teams’ collaborative making processes. International Journal of Computer-Supported Collaborative Learning, 15(3), 319–349. https://doi.org/10.1007/s11412-020-09330-6 Roehler, L. R., & Cantlon, D. J. (1997). Scaffolding: A powerful tool in social constructivist classrooms. In K. Hogan & M. Pressley (Eds.), Scaffolding student learning: Instructional approaches and issues (pp. 6–42). Brookline. Roschelle, J., & Teasley, S. (1995). The construction of shared knowledge in collaborative problem solving. In C. O’Malley (Ed.), Computer-supported collaborative learning (pp. 69–197). Springer. Samuelsen, J., Chen, W., & Wasson, B. (2019). Integrating multiple data sources for learning analytics—Review of literature. Research and Practice in Technology Enhanced Learning, 14(1), 1–20. https://doi.org/10.1186/s41039-019-0105-4 Sandoval, W. A., & Reiser, B. J. (2004). Explanation-driven inquiry: Integrating conceptual and epistemic scaffolds for scientific inquiry. Science Education, 88(3), 345–372. https://doi.org/10.1002/sce.10130 Schneider, B., Dowell, N., & Thompson, K. (2021). Collaboration analytics—current state and potential futures. Journal of Learning Analytics, 8(1), 1–12. Serrano-Cámara, L. M., Paredes-Velasco, M., Alcover, C. M., & Velazquez-Iturbide, J. Á. (2014). An evaluation of students’ motivation in computer-supported collaborative learning of programming concepts. Computers in Human Behavior, 31, 499–508. https://doi.org/10.1016/j.chb.2013.04.030 Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10.18608/jla.2016.33.3 Shaffer, D. W., Hatfield, D. L., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., Frank, K. A., Rupp, A. A., & Mislevy, R. (2009). Epistemic network analysis: A prototype for 21st century assessment of learning. International Journal of Learning and Media, 1(2), 1–21. https://doi.org/10.1162/ijlm.2009.0013 Shaffer, D. W., & Ruis, A. R. (2017). Epistemic network analysis: A worked example of theory-based learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (pp. 175–187). Society for Learning Analytics and Research. Sinha, S., Rogat, T. K., Adams-Wiggins, K. R., & Hmelo-Silver, C. E. (2015). Collaborative group engagement in a computer-supported inquiry learning environment. International Journal of Computer-Supported Collaborative Learning, 10(3), 273–307. https://doi.org/10.1007/s11412-015-9218-y Socratous, C., & Ioannou, A. (2021). Structured or unstructured educational robotics curriculum? A study of debugging in block-based programming. Educational Technology Research and Development, 69(6), 3081–3100. https://doi.org/10.1007/s11423-021-10056-x Socratous, C., & Ioannou, A. (2022). Evaluating the impact of the curriculum structure on group metacognition during collaborative problem-solving using educational robotics. TechTrends. https://doi.org/10.1007/s11528-022-00738-5 Stahl, G. (2009). Studying virtual math teams. New York: Springer. Available at http://GerryStahl.net/vmt/book. Sun, D., Ouyang, F., Li, Y., & Zhu, C. (2021). Comparing learners’ knowledge, behaviors, and attitudes between two instructional modes of computer programming in secondary education. International Journal of STEM Education, 8(54), 1–15. https://doi.org/10.1186/s40594-021-00311-1 Sun, J. C., & Hsu, K. Y. (2019). A smart eye-tracking feedback scaffolding approach to improving students’ learning self-efficacy and performance in a C programming course. Computers in Human Behavior, 95, 66–72. https://doi.org/10.1016/j.chb.2019.01.036 Tartas, V., & Perret-Clermont, A. N. (2008). Socio-cognitive dynamics in dyadic interaction: How do you work together to solve Kohs cubes? European Journal of Developmental Psychology, 5(5), 561–584. https://doi.org/10.1080/17405620701859522 Tawfik, A. A., Law, V., Ge, X., Xing, W., & Kim, K. (2018). The effect of sustained vs. faded scaffolding on students’ argumentation in ill-structured problem solving. Computers in Human Behavior, 87, 436–449. https://doi.org/10.1016/j.chb.2018.01.035 Teague, D., & Roe, P. (2008). Collaborative learning—Towards a solution for novice programmers. In S. Hamilton & M. Hamilton (Eds.), Proceedings of the tenth conference on Australasian computing education—CRPIT Volume 78 (pp. 147–153). Australian Computer Society, Australia. Tohyama, S., Matsuzawa, Y., Yokoyama, S., Koguchi, T., & Takeuchi, Y. (2018). Constructive interaction on collaborative programming: Case study for grade 6 students group. IFIP world conference on computers in education (pp. 589–598). Springer International Publishing. https://doi.org/10.1007/978-3-319-74310-3_59 Twiner, A., Littleton, K., Whitelock, D., & Coffin, C. (2021). Combining sociocultural discourse analysis and multimodal analysis to explore teachers’ and pupils’ meaning making. Learning, Culture and Social Interaction, 30, 100520. https://doi.org/10.1016/j.lcsi.2021.100520 Umapathy, K., & Ritzhaupt, A. D. (2017). A meta-analysis of pair-programming in computer programming courses: Implications for educational practice. ACM Transactions on Computing Education, 17(4), 1–13. https://doi.org/10.1145/2996201 van de Pol, J., Mercer, N., & Volman, M. (2019). Scaffolding student understanding in small-group work: Students’ uptake of teacher support in subsequent small-group interaction. Journal of the Learning Sciences, 28(2), 206–239. https://doi.org/10.1080/10508406.2018.1522258 van de Pol, J., Volman, M., & Beishuizen, J. (2010). Scaffolding in teacher–student interaction: A decade of research. Educational Psychology Review, 22(3), 271–296. https://doi.org/10.1007/s10648-010-9127-6 van de Pol, J., Volman, M., Oort, F., & Beishuizen, J. (2015). The effects of scaffolding in the classroom: Support contingency and student independent working time in relation to student achievement, task effort and appreciation of support. Instructional Science, 43(5), 615–641. https://doi.org/10.1007/s11251-015-9351-z Veerasamy, A. K., D’Souza, D., Linden, R., & Laakso, M. J. (2019). Relationship between perceived problem-solving skills and academic performance of novice learners in introductory programming courses. Journal of Computer Assisted Learning, 35, 246–255. https://doi.org/10.1111/jcal.12326 Villamor, M. M., & Rodrigo, M. M. T. (2019). Gaze collaboration patterns of successful and unsuccessful programming pairs using cross-recurrence quantification analysis. Research and Practice in Technology Enhanced Learning, 14(1), 1–22. https://doi.org/10.1186/s41039-019-0118-z Wang, X., & Hwang, G. (2017). A problem posing-based practicing strategy for facilitating students’ computer programming skills in the team-based learning mode. Educational Technology Research and Development, 65(6), 1655–1671. https://doi.org/10.1007/s11423-017-9551-0 Webb, N. M. (2009). The teacher’s role in promoting collaborative dialogue in the classroom. British Journal of Educational Psychology, 79, 1–28. https://doi.org/10.1348/000709908X380772 Webb, N. M., & Farivar, S. (1999). Developing productive group interaction in middle school. In A. M. O’Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 117–149). Lawrence Erlbaum Associates. Webb, N. M., Nemer, K. M., & Ing, M. (2006). Small group reflections: Parallels between teacher discourse and student behavior in peer-directed groups. Journal of the Learning Sciences, 15(1), 63–119. https://doi.org/10.1207/s15327809jls1501_8 Wei, X., Lin, L., Meng, N., Tan, W., & Kong, S. C. (2020). The effectiveness of partial pair programming on elementary school students’ computational thinking skills and self-efficacy. Computers & Education, 160, 104023. https://doi.org/10.1016/j.compedu.2020.104023 Wiig, C., Silseth, K., & Erstad, O. (2017). Creating intercontextuality in students learning trajectories. Opportunities and difficulties. Language and Education, 32(1), 43–59. https://doi.org/10.1080/09500782.2017.1367799 Wittwer, J., Nückles, M., Landmann, N., & Renkl, A. (2010). Can tutors be supported in giving effective explanations? Journal of Educational Psychology, 102(1), 74–89. https://doi.org/10.1037/a0016727 Wittwer, J., & Renkl, A. (2008). Why instructional explanations often do not work: A framework for understanding the effectiveness of instruction explanations. Educational Psychologist, 43(1), 49–64. https://doi.org/10.1080/00461520701756420 Wu, B., Hu, Y., Ruis, A. R., & Wang, M. (2019). Analysing computational thinking in collaborative programming: A quantitative ethnography approach. Journal of Computer Assisted Learning, 35(3), 421–434. https://doi.org/10.1111/jcal.12348 Yeomans, L., Zschaler, S., & Coate, K. (2019). Transformative and troublesome? Students’ and professional programmers’ perspectives on difficult concepts in programming. ACM Transactions on Computing Education, 19(3), 1–27. https://doi.org/10.1145/3283071 Zatarain Cabada, R., Barrón Estrada, M. L., Ríos Félix, J. M., & Alor Hernández, G. (2018). A virtual environment for learning computer coding using gamification and emotion recognition. Interactive Learning Environments, 28(8), 1048–1063. https://doi.org/10.1080/10494820.2018.1558256 Zhang, J. (2007). A cultural look at information and communication technologies in Eastern education. Educational Technology Research and Development, 55(3), 301–314. https://doi.org/10.1007/s11423-007-9040-y Zhang, J. (2013). Chapter 28: Collaboration, technology, and culture. In C. E. Hmelo-Silver, C. A. Chinn, C. K. K. Chan, & A. M. O’Donnell (Eds.), The international handbook of collaborative learning (pp. 495–508). Routledge. Zheng, L. (2021). Improving programming skills through an innovative collaborative programming model: A case study. Lecture notes in educational technologyData-driven design for computer-supported collaborative learning (pp. 75–85). Springer. https://doi.org/10.1007/978-981-16-1718-8_6 Zheng, L., Zhen, Y., Niu, J., & Zhong, L. (2022). An exploratory study on fade-in versus fade-out scaffolding for novice programmers in online collaborative programming settings. Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-021-09307-w