Framework for automatically suggesting remedial actions to help students at risk based on explainable ML and rule-based models
Tóm tắt
Higher education institutions often struggle with increased dropout rates, academic underachievement, and delayed graduations. One way in which these challenges can potentially be addressed is by better leveraging the student data stored in institutional databases and online learning platforms to predict students’ academic performance early using advanced computational techniques. Several research efforts have focused on developing systems that can predict student performance. However, there is a need for a solution that can predict student performance and identify the factors that directly influence it. This paper aims to develop a model that accurately identifies students who are at risk of low performance, while also delineating the factors that contribute to this phenomenon. The model employs explainable machine learning (ML) techniques to delineate the factors that are associated with low performance and integrates rule-based model risk flags with the developed prediction system to improve the accuracy of performance predictions. This helps low-performing students to improve their academic metrics by implementing remedial actions that address the factors of concern. The model suggests proper remedial actions by mapping the students’ performance in each identified checkpoint with the course learning outcomes (CLOs) and topics taught in the course. The list of possible actions is mapped to this checkpoint. The developed model can accurately distinguish students at risk (total grade
$$< 70\%$$
) from students with good performance. The Area under the ROC Curve (AUC ROC) of binary classification model fed with four checkpoints reached 1.0. Proposed framework may aid the student to perform better, increase the institution’s effectiveness and improve their reputations and rankings.
Tài liệu tham khảo
Al-Rahmi, W., Aldraiweesh, A., Yahaya, N., Kamin, Y. B., & Zeki, A. M. (2019). Massive open online courses (moocs): Data on higher education. Data in Brief, 22, 118–125.
Alapont, J., Bella-Sanjuán, A., Ferri, C., Hernández-Orallo, J., Llopis-Llopis, J., & Ramírez-Quintana, M. (2005). Specialised tools for automating data mining for hospital management. In: Proceedings of First East European Conference on Health Care Modelling and Computation, pp 7–19.
Alboaneen, D., Almelihi, M., Alsubaie, R., Alghamdi, R., Alshehri, L., & Alharthi, R. (2022). Development of a web-based prediction system for students’ academic performance. Data, 7(2), 21.
Albreiki, B., Habuza, T., Shuqfa, Z., Serhani, M. A., Zaki, N., & Harous, S. (2021). Customized rule-based model to identify at-risk students and propose rational remedial actions. Big Data and Cognitive Computing, 5(4), 71.
Albreiki, B., Zaki, N., & Alashwal, H. (2021). A systematic literature review of student’performance prediction using machine learning techniques. Education Sciences, 11(9), 552.
Alhassan, A., Zafar, B., & Mueen, A. (2020). Predict students’ academic performance based on their assessment grades and online activity data. International Journal of Advanced Computer Science and Applications (IJACSA) 11(4), 185–194.
Altujjar, Y., Altamimi, W., Al-Turaiki, I., & Al-Razgan, M. (2016). Predicting critical courses affecting students performance: A case study. Procedia Computer Science, 82, 65–71.
Alturki, R. A., et al. (2016). Measuring and improving student performance in an introductory programming course. Informatics in Education-An International Journal, 15(2), 183–204.
Bengio, Y., Lecun, Y., & Hinton, G. (2021). Deep learning for AI. Communications of the ACM, 64(7), 58–65.
Borrella, I., Caballero-Caballero, S., & Ponce-Cueto, E. (2022). Taking action to reduce dropout in moocs: Tested interventions. Computers & Education, 179(104), 412.
Buenaño-Fernández, D., Gil, D., & Luján-Mora, S. (2019). Application of machine learning in predicting performance for computer engineering students: A case study. Sustainability, 11(10), 2833.
Cornell-Farrow, S., & Garrard, R. (2020). Machine learning classifiers do not improve the prediction of academic risk: Evidence from australia. Communications in Statistics: Case Studies, Data Analysis and Applications, 6(2), 228–246.
Costa, E. B., Fonseca, B., Santana, M. A., de Araújo, F. F., & Rego, J. (2017). Evaluating the effectiveness of educational data mining techniques for early prediction of students’ academic failure in introductory programming courses. Computers in Human Behavior, 73, 247–256.
Dekker, I., De Jong, E. M., Schippers, M. C., Bruijn-Smolders, D., Alexiou, A., Giesbers, B., et al. (2020). Optimizing students’ mental health and academic performance: AI-enhanced life crafting. Frontiers in Psychology, 11, 1063.
Evangelista, E. (2021). A hybrid machine learning framework for predicting students’ performance in virtual learning environment. International Journal of Emerging Technologies in Learning (iJET) 16(24), 255–272.
Fahd, K., Venkatraman, S., Miah, S. J., & Ahmed, K. (2022). Application of machine learning in higher education to assess student academic performance, at-risk, and attrition: A meta-analysis of literature. Education and Information Technologies, 27, 3743–3775.
Goga, M., Kuyoro, S., & Goga, N. (2015). A recommender for improving the student academic performance. Procedia-Social and Behavioral Sciences, 180, 1481–1488.
Gong, B., Nugent, J. P., Guest, W., Parker, W., Chang, P. J., Khosa, F., & Nicolaou, S. (2019). Influence of artificial intelligence on canadian medical students’ preference for radiology specialty: Anational survey study. Academic Radiology, 26(4), 566–577.
Gupta, S. K., Antony, J., Lacher, F., & Douglas, J. (2020). Lean six sigma for reducing student dropouts in higher education-an exploratory study. Total Quality Management & Business Excellence, 31(1–2), 178–193.
Ha, D. T., Loan, P. T. T., Giap, C. N., & Huong, N. . TL. (2020). An empirical study for student academic performance prediction using machine learning techniques. International Journal of Computer Science and Information Security (IJCSIS) 18(3), 21–28.
Hasan, R., Palaniappan, S., Mahmood, S., Abbas, A., Sarker, K. U., & Sattar, M. U. (2020). Predicting student performance in higher educational institutions using video learning analytics and data mining techniques. Applied Sciences, 10(11), 3894.
Hernández-Blanco, A., Herrera-Flores, B., Tomás, D., & Navarro-Colorado, B. (2019). A systematic review of deep learning approaches to educational data mining. Complexity, 2019(1):1–22.
Hu, Y. H., Lo, C. L., & Shih, S. P. (2014). Developing early warning systems to predict students’ online learning performance. Computers in Human Behavior, 36, 469–478.
Hussain, M., Zhu, W., Zhang, W., & Abidi, SMR. (2018). Student engagement predictions in an e-learning system and their impact on student course assessment scores. Computational Intelligence and Neuroscience. https://doi.org/10.1155/2018/6347186
Iatrellis, O., Savvas, I. K., Fitsilis, P., & Gerogiannis, V. C. (2021). A two-phase machine learning approach for predicting student outcomes. Education and Information Technologies, 26(1), 69–88.
Inyang, U. G., Eyoh, I. J., Robinson, S. A., & Udo, E. N. (2019). Visual association analytics approach to predictive modelling of students’ academic performance. International Journal of Modern Education & Computer Science 11(12), 1–13.
Koprinska, I., Stretton, J., & Yacef, K. (2015). Students at risk: Detection and remediation. In: EDM, pp 512–515.
Kruck, S., & Lending, D. L. D. (2003). Predicting academic performance in an introductory college introductory college-level is course level is course. Information Technology, Learning, and Performance Journal, 21(2), 9.
Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z., Vaclavek, J., & Wolff, A. (2015). Ou analyse: Analysing at-risk students at the open university. Learning Analytics Review, 1–16.
Li, K., Uvah, J., & Amin, R. (2012). Predicting students’ performance in elements of statistics. Online Submission, 10, 875–884.
Liao, S. N., Zingaro, D., Thai, K., Alvarado, C., Griswold, W. G., & Porter, L. (2019). A robust machine learning technique to predict low-performing students. ACM Transactions on Computing education (TOCE), 19(3), 1–19.
Lykourentzou, I., Giannoukos, I., Mpardis, G., Nikolopoulos, V., & Loumos, V. (2009). Early and dynamic student achievement prediction in e-learning courses using neural networks. Journal of the American Society for Information Science and Technology, 60(2), 372–380.
Marbouti, F., Diefes-Dux, H. A., & Madhavan, K. (2016). Models for early prediction of at-risk students in a course using standards-based grading. Computers & Education, 103, 1–15.
Moonsamy, D., Naicker, N., Adeliyi, TT., & Ogunsakin, R. E. (2021). A meta-analysis of educational data mining for predicting students performance in programming. International Journal of Advanced Computer Science and Applications, 12(2), 97–104.
Mousavinasab, E., Zarifsanaiey, N., Niakan Kalhori, R. S., Rakhshan M., Keikha L., & Ghazi Saeedi M. (2021). Intelligent tutoring systems: a systematic review of characteristics, applications, and evaluation methods. Interactive Learning Environments, 29(1), 142–163.
Muñoz-Carpio, J. C., Jan, Z., & Saavedra, A. (2021). Machine learning for learning personalization to enhance student academic performance. In: LALA, pp 88–99.
Namoun, A., & Alshanqiti, A. (2021). Predicting student performance using data mining and learning analytics techniques: A systematic literature review. Applied Sciences, 11(1), 237.
Prenkaj, B., Velardi, P., Stilo, G., Distante, D., & Faralli, S. (2020). A survey of machine learning approaches for student dropout prediction in online courses. ACM Computing Surveys (CSUR), 53(3), 1–34.
Purwaningsih, N., & Arief, D. R. (2018). Predicting students’ performance in english class. In: AIP Conference Proceedings, AIP Publishing LLC, vol 1977, p 020020.
Qazdar, A., Er-Raha, B., Cherkaoui, C., & Mammass, D. (2019). A machine learning algorithm framework for predicting students performance: a case study of baccalaureate students in morocco. Education and Information Technologies, 24(6), 3577–3589.
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016).“why should i trust you?” explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135–1144.
Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12–27.
Romero, C., Espejo, P. G., Zafra, A., Romero, J. R., & Ventura, S. (2013). Web usage mining for predicting final marks of students that use moodle courses. Computer Applications in Engineering Education, 21(1), 135–146.
Sarker, F., Tiropanis, T., & Davis, HC. (2013). Students’ performance prediction by using institutional internal and external open data sources. eprintssotonacuk.
Shahiri, A. M., Husain, W., et al. (2015). A review on predicting student’s performance using data mining techniques. Procedia Computer Science, 72, 414–422.
Tomasevic, N., Gvozdenovic, N., & Vranes, S. (2020). An overview and comparison of supervised data mining techniques for student exam performance prediction. Computers & education, 143(103), 676.
Urkude, S., & Gupta, K. (2019). Student intervention system using machine learning techniques. International Journal of Engineering and Advanced Technology, 8(6), 21–29.
Wagner, E. P., Sasser, H., & DiBiase, W. J. (2002). Predicting students at risk in general chemistry using pre-semester assessments and demographic information. Journal of Chemical Education, 79(6), 749.
Waheed, H., Hassan, S. U., Aljohani, N. R., Hardman, J., Alelyani, S., & Nawaz, R. (2020). Predicting academic performance of students from vle big data using deep learning models. Computers in Human behavior, 104(106), 189.
Watson, C., Li, F. W., & Godwin, J. L. (2013). Predicting performance in an introductory programming course by logging and analyzing student programming behavior. In: 13 IEEE 13th international conference on advanced learning technologies, IEEE, pp 319–323.
Wolff, A., Zdrahal, Z., Nikolov, A., & Pantucek, M. (2013). Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment. In: Proceedings of the third international conference on learning analytics and knowledge, pp 145–149.
Xing, W., Guo, R., Petakovic, E., & Goggins, S. (2015). Participation-based student final performance prediction model through interpretable genetic programming: Integrating learning analytics, educational data mining and theory. Computers in Human Behavior, 47, 168–181.
Yadav, S. K., Bharadwaj, B., & Pal, S. (2012). Data mining applications: A comparative study for predicting student’s performance. arXiv preprint arXiv:1202.4815
Yukselturk, E., Ozekes, S., & Türel, Y. K. (2014). Predicting dropout student: An application of data mining methods in an online education program. European Journal of Open, Distance and e-learning, 17(1), 118–133.
Zhao, Q., Wang, J. L., Pao, T. L., & Wang, L. Y. (2020). Modified fuzzy rule-based classification system for early warning of student learning. Journal of Educational Technology Systems, 48(3), 385–406.