Optimising Moodle quizzes for online assessments
Tóm tắt
Computer-aided learning management systems (LMSs) are widely used in higher education and are viewed as beneficial when transitioning from conventional face-to-face teaching to fully online courses. While LMSs have unique tools for transferring and assessing knowledge, their ability to engage and assess learners needs further investigation. This paper focuses on a study examining the LMS “Moodle” to ascertain the effectiveness of “Moodle quizzes” to improve, assess and distinguish knowledge in a civil engineering course at an Australian university. The course has a database comprising 62 formative and 61 summative quiz questions with embedded text, images, audio and video. This study investigates the use of these quiz questions with four course cohorts and 169 students. The quizzes assessed competencies of students during various stages of a study period through automated marking. The suitability of questions to assess and distinguish student knowledge levels was determined using a psychometric analysis based on facility index (FI) and the discrimination index (DI) statistics embedded within the Moodle quizzes. This study highlights strategies used to set and review quiz questions for formative and summative assessments. Results indicated that students were engaged and satisfied in the formative assessment because they viewed the interactive videos between 2 and 6 times and 65% of students attempted all the formative questions. The FI indicated student pass rate for the summative questions and DI indicated the difficulty of these questions, while the combination of FI and DI results separated students with different knowledge levels. Using these Moodle statistics provided information to make effective decisions on how to improve the summative quizzes. The multimodal quizzes were effective in teaching and assessing a theoretical engineering course and provided efficient methods to replace conventional assessments. The FI and DI indexes are useful statistical tools in redesigning appropriate sets of questions. Time-poor academics will benefit from using these easily attainable Moodle statistics to inform decisions while revising the quizzes and making assessments more autonomous.
Tài liệu tham khảo
Andre, L. (2019). 20 best LMS software solutions of 2019, FinancesOnline. All B2B. https://financesonline.com/top-20-lms-software-solutions/. Accessed 9 June 2019.
Aydin, C. C., & Tirkes, G. (2010). Open source learning management systems in distance learning. Turkish Online Journal of Educational Technology, 9(2), 175–184.
Blanco, M., & Ginovart, M. (2010). Moodle quizzes for assessing statistical topics in engineering studies. Proceedings of the the Joint International IGIP-SEFI Annual Conference, September 19-22, 2010. https://upcommons.upc.edu/bitstream/handle/2117/9992/blanco_ginovart_igip_sefi_2010.pdf. Accessed 8 June 2019.
Broadbent, J., & Fuller-Tyszkiewicz, M. (2018). Profiles in self-regulated learning and their correlates for online and blended learning students. A bi-monthly publication of the Association for Educational Communications & Technology, 66(6), 1435–1455. https://doi.org/10.1007/s11423-018-9595-9.
Butcher, P. (2010). Quiz report statistics. Moodle. https://docs.moodle.org/dev/Quiz_report_statistics. Accessed 26 Feb 2019.
Cohen, D., & Sasson, I. (2016). Online quizzes in a virtual learning environment as a tool for Formative Assessment. Journal of Technology and Science Education, 6(3), 188–208. https://doi.org/10.3926/jotse.217.
Cook, B. R., & Babon, A. (2017). Active learning through Online Quizzes: Better learning and less (busy) work. Journal of Geography in Higher Education, 41(1), 24–38. https://doi.org/10.1080/03098265.2016.1185772.
eLearning Industry (2018). Top LMS software based on customer experience. https://elearningindustry.com/directory/software-categories/learning-management-systems/best/customer-experience. Accessed 9 June 2019.
Ennis, R. H. (1993). Critical thinking assessment. Theory Into Practice, 32(3), 179–186. https://doi.org/10.1080/00405849309543594.
Fales-Williams, A., Kramer, T., Heer, R., & Danielson, J. (2005). A quiz becomes a multidirectional dialogue with web-based instructional tools for an anatomical pathology rotation. Journal of Veterinary Medical Education, 32(1), 144–149. https://doi.org/10.3138/jvme.32.1.144.
Francis, R., & Shannon, S. J. (2013). Engaging with blended learning to improve students’ learning outcomes. European Journal of Engineering Education, 38(4), 359–369. https://doi.org/10.1080/03043797.2013.766679.
Geary, D. C. (2017). Acquisition of complex arithmetic skills and higher order mathematics concepts. London: Elsevier.
Gómez-Soberón, J. M., Gómez-Soberón, M. C., Corral-Higuera, R., Arredondo-Rea, S. P., Almaral-Sánchez, J. L., & Cabrera-Covarrubias, F. G. (2013). Calibrating questionnaires by psychometric analysis to evaluate knowledge. SAGE Open, 3(3), 1–14. https://doi.org/10.1177/2158244013499159.
Heo, M., & Chow, A. (2005). The impact of computer augmented online learning and assessment tool. Educational Technology & Society, 8(1), 113–125.
Huang, S., & Fang, N. (2013). Predicting student academic performance in an engineering dynamics course: A comparison of four types of predictive mathematical models. Computers & Education, 61(1), 133–145. https://doi.org/10.1016/j.compedu.2012.08.015.
Jackson, B. L. (2017). Higher education faculty utilization of online technological tools: A multilevel analysis. Journal of Educational Multimedia and Hypermedia, 26(3), 271–283.
Jaeger, M., & Adair, D. (2017). Time pressure in scenario-based online construction safety quizzes and its effect on students’ performance. European Journal of Engineering Education, 42(3), 241–251. https://doi.org/10.1080/03043797.2016.1153042.
James, R. (2016). Tertiary student attitudes to invigilated, online summative examinations. International Journal of Educational Technology in Higher Education, 13(1), 1–13. https://doi.org/10.1186/s41239-016-0015-0.
Krause, C., Krause, R., Krause, R., Gomez, N., Jafry, Z., & Dinh, V. A. (2017). Effectiveness of a 1-hour extended focused assessment with sonography in trauma session in the medical student surgery clerkship. Journal of Surgical Education, 74(6), 968–974. https://doi.org/10.1016/j.jsurg.2017.03.007.
Liagkou, V., & Stylios, C. (2018). A trustworthy and privacy preserving model for online competence evaluation system. In International conference on dependability and complex Systems (pp. 338–347). Springer. https://doi.org/10.1007/978-3-319-91446-6.
Maarek, J. (2018). Benefits of active learning embedded in online content material supporting a flipped classroom. Proceedings fo the ASEE Annual Conference & Exposition, Salt Lake City. https://peer.asee.org/29845. Accessed 14 June 2019.
Martins, S. G. (2017). Weekly online quizzes to a mathematics course for engineering students. Teaching Mathematics and its Applications: An International Journal of the IMA, 36(1), 56–63. https://doi.org/10.1093/teamat/hrw011.
Moodle Statistics. (2019a). Moodle. https://docs.moodle.org/dev/Quiz_report_statistics. Accessed 26 Feb 2019.
Moodle Statistics. (2019b). Moodle. https://docs.moodle.org/dev/Quiz_statistics_calculations#Introduction. Accessed 21 July 2019.
Philipsen, B., Tondeur, J., Roblin, N. P., Vanslambrouck, S., & Zhu, C. (2019). Improving teacher professional development for online and blended learning: A systematic meta-aggregative review. Educational Technology Research and Development, 1–30. https://doi.org/10.1007/s11423-019-09645-8.
Ransdell, S. (2001). Predicting college success: the importance of ability and non-cognitive variables. International Journal of Educational Research, 35(4), 357–364. https://doi.org/10.1016/S0883-0355(01)00032-5.
Rinaldi, V. D., Lorr, N. A., & Williams, K. (2017). Evaluating a technology supported interactive response system during the laboratory section of a histology course. Anatomical Sciences Education, 10(4), 328–338. https://doi.org/10.1002/ase.1667.
Root-Kustritz, M. V. (2014). Canine theriogenology for dog enthusiasts: teaching methodology and outcomes in a massive open online course (MOOC). Journal of Veterinary Medical Education, 41(1), 9–18. https://doi.org/10.3138/jvme.0813-112R1.
Salamon, H., Ali, N., Miskon, S., & Ahmad, N. (2016). Initial recommendations of MOOCs characteristics for academic discipline clusters. Journal of Theoretical and Applied Information Technology, 87(2), 204–213.
Sancho-Vinuesa, T., Masià, R., Fuertes-Alpiste, M., & Molas-Castells, N. (2018). Exploring the effectiveness of continuous activity with automatic feedback in online calculus. Computer Applications in Engineering Education, 26(1), 62–74. https://doi.org/10.1002/cae.21861.
Schneider, J. L., Ruder, S. M., & Bauer, C. F. (2018). Student perceptions of immediate feedback testing in student centered chemistry classes. Chemistry Education Research and Practice, 19(2), 442–451. https://doi.org/10.1039/c7rp00183e.
Sullivan, D. P. (2016). An integrated approach to preempt cheating on asynchronous, objective, online assessments in graduate business classes. Online Learning, 20(3), 195–209. https://doi.org/10.24059/olj.v20i3.650.
Thevenot, C. (2010). Arithmetic word problem solving: Evidence for the construction of a mental model. Acta Psychologica, 133(1), 90–95. https://doi.org/10.1016/j.actpsy.2009.10.004.
Thevenot, C., Devidal, M., Barrouillet, P., & Fayol, M. (2007). Why does placing the question before an arithmetic word problem improve performance? A situation model account. Quarterly Journal of Experimental Psychology, 60(1), 43–56. https://doi.org/10.1080/17470210600587927.
Ting, S. M. R. (2001). Predicting academic success of first-year engineering students from standardized test scores and psychosocial variables. International Journal of Engineering Education, 17(1), 75–80.
Vicente, S., Orrantia, J., & Verschaffel, L. (2008). Influence of mathematical and situational knowledge on arithmetic word problem solving: Textual and graphical aids. Infancia Y Aprendizaje, 31(4), 463–483. https://doi.org/10.1174/021037008786140959.
Wallihan, R., Smith, K. G., Hormann, M. D., Donthi, R. R., Boland, K., & Mahan, J. D. (2018). Utility of intermittent online quizzes as an early warning for residents at risk of failing the pediatric board certification examination.(Report). BMC Medical Education, 18, 1. https://doi.org/10.1186/s12909-018-1366-0.
West, J., & Turner, W. (2016). Enhancing the assessment experience: Improving student perceptions, engagement and understanding using online video feedback. Innovations in Education and Teaching International, 53(4), 400–410. https://doi.org/10.1080/14703297.2014.1003954.
Williams van Rooij, S. (2012). Open-source learning management systems: A predictive model for higher education. Journal of Computer Assisted Learning, 28(2), 114–125. https://doi.org/10.1111/j.1365-2729.2011.00422.x.
Wojcikowski, K., & Kirk, L. (2013). Immediate detailed feedback to test-enhanced learning: An effective online educational tool. Medical Teacher, 35(11), 915–919. https://doi.org/10.3109/0142159X.2013.826793.