The relationship between differences in students’ computer and information literacy and response times: an analysis of IEA-ICILS data
Tóm tắt
Due to the increasing use of information and communication technology, computer-related skills are important for all students in order to participate in the digital age (Fraillon, J., Ainley, J., Schulz, W., Friedman, T. & Duckworth, D. (2019). Preparing for life in a digital world: IEA International Computer and Information Literacy Study 2018 International Report. Amsterdam: International Association for the Evaluation of Educational Achievement (IEA). Retrieved from
https://www.iea.nl/sites/default/files/2019-11/ICILS%202019%20Digital%20final%2004112019.pdf
). Educational systems play a key role in the mediation of these skills (Eickelmann. Second Handbook of Information Technology in Primary and Secondary Education. Cham: Springer, 2018). However, previous studies have shown differences in students’ computer and information literacy (CIL). Although various approaches have been used to explain these differences, process data, such as response times, have never been taken into consideration. Based on data from the IEA-study ICILS 2013 of the Czech Republic, Denmark and Germany, this secondary analysis examines to what extent response times can be used as an explanatory approach for differences in CIL also within different groups of students according to student background characteristics (gender, socioeconomic background and immigrant background). First, two processing profiles using a latent profile analysis (Oberski, D. (2016). Mixture Models: Latent Profile and Latent Class Analysis. In J. Robertson & M. Kaptein (Eds.), Modern Statistical Methods for HCI (pp. 275–287). Switzerland: Springer.
https://doi.org/10.1007/978-3-319-26633-6
) based on response times are determined—a fast and a slow processing profile. To detect how these profiles are related to students’ CIL, also in conjunction with students’ background characteristics (socioeconomic and immigrant background), descriptive statistics are used. The results show that in the Czech Republic and Germany, students belonging to the fast processing profile have on average significantly higher CIL than students allocated to the slow processing profile. In Denmark, there are no significant differences. Concerning the student background characteristics in the Czech Republic, there are significant negative time-on-task effects for all groups except for students with an immigrant background and students with a high parental occupational status. There are no significant differences in Denmark. For Germany, a significant negative time-on-task effect can be found among girls. However, the other examined indicators for Germany are ambiguous. The results show that process data can be used to explain differences in students’ CIL: In the Czech Republic and Germany, there is a correlation between response times and CIL (significant negative time-on-task effect). Further analysis should also consider other aspects of CIL (e.g. reading literacy). What becomes clear, however, is that when interpreting and explaining differences in competence, data should also be included that relates to the completion process during testing.
Tài liệu tham khảo
ACARA. (2015). National Assessment Program – ICT Literacy. Years 6 & 10. Report 2014. Sydney: ACARA. Retrieved from https://www.nap.edu.au/_resources/D15_8761__NAP-ICT_2014_Public_Report_Final.pdf
ACARA. (2018). NAP Sample Assessment ICT Literacy. Years 6 and 10. Sydney: ACARA. Retrieved from https://www.nap.edu.au/docs/defaultsource/default-document-library/2017napictlreport_final.pdf?sfvrsn=b5696d5e_2
Aesaert, K. & van Braak, J. (2018). Information and Communication Competences for Students. In J. Voogt; G. Knezek, R. Christensen & K.-W. Lai (Eds.), Second Handbook of Information Technology in Primary and Secondary Education (pp. 255–269). Cham: Springer. https://doi.org/10.1007/978-3-319-53803-7
Aesaert, K., & van Braak, J. (2015). Gender and socioeconomic related differences in performance based ICT competences. Computers & Education, 84, 8–25. https://doi.org/10.1016/j.compedu.2014.12.017
Eickelmann, B. (2018). Section introduction: international policies on information and communication technology in primary and secondary schools. In J. Voogt, G. Knezek, R. Christensen & K.-W. Lai (Eds.), Second Handbook of Information Technology in Primary and Secondary Education. Cham: Springer. https://doi.org/10.1007/978-3-319-53803-7
Gerick, J., Eickelmann, B. & Bos, W. (2017). The international computer and information literacy study from a European perspective: introduction to the special issue. European Educational Research Journal, 16(6), 707–715. https://doi.org/10.1177/1474904117735417
Bonfadelli, H., Bucher, P., & Piga, A. (2007). Use of old and new media in ethnic minority youth in Europe with a special emphasis on Switzerland. Communications, 32(2), 141–170. https://doi.org/10.1515/COMMUN.2007.010
D’Haenens, L. (2003). ICT in multicultural society The Netherlands: A context for sound multiform media policy? Gazette, 65(4–5), 401–421. https://doi.org/10.1177/0016549203654006
Fraillon, J., Ainley, J., Schulz, W., Friedman, T. & Duckworth, D. (2019). Preparing for life in a digital world: IEA International Computer and Information Literacy Study 2018 International Report. Amsterdam: International Association for the Evaluation of Educational Achievement (IEA). Retrieved from https://www.iea.nl/sites/default/files/2019-11/ICILS%202019%20Digital%20final%2004112019.pdf
Fraillon, J., Schulz, W., Friedman, T., Ainley, J., & Gebhardt, E. (2015). ICILS 2013 Technical Report. Amsterdam: IEA Secretariat. https://doi.org/10.15478/uuid:b9cdd888-6665-4e9f-a21e-61569845ed5b
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for Life in a Digital Age. The IEA International Computer and Information Literacy Study International Report. Melbourne: Australian Council for Educational Research (ACER). https://doi.org/10.1007/978-3-319-14222-7
Ganzeboom, H. B. G., de Graaf, P. M., & Treiman, D. J. (1992). A standard international socio-economic index of occupational status. Social Science Research, 21(1), 1–56.
Gebhardt, E., Thomson, S., Ainley, J., & Hillman, K. (2019). Gender differences in computer and information literacy. An In-depth Analysis of Data from ICILS. Cham: Springer. https://doi.org/10.1007/978-3-030-26203-7_1
Gerick, J. (2018). School Level Characteristics and Students’ CIL in Europe—a latent class analysis approach. Computers & Education, 120, 160–171. https://doi.org/10.1016/j.compedu.2018.01.013
Goldhammer, F. (2013). Prozessbezogene Verhaltensdaten für Rückmeldung in digitalen Lernumgebungen. [Process-related behavioral data for feedback in digital learning environments.] Deutsches Institut für Internationale Pädagogische Forschung (DIPF). Zentrum für internationale Bildungsvergleichsstudien (ZIB).
Goldhammer, F., Naumann, J., Rölke, H., Stelter, A. & Tóth, K. (2017). Relating Product Data to Process Data from Computer-Based Competency Assessment. In D. Leutner, J. Fleischer, J. Grünkorn & E. Klieme (Eds.), Competence Assessment in Education. Research, Models and Instruments (pp. 407–425). Heidelberg: Springer. https://doi.org/10.1007/978-3-319-50030-0
Goldhammer, F., Naumann, J., Stelter, A., Rölke, H., Tóth, K., & Klieme, E. (2014). The time-on-task effect in reading and problem solving is moderated by item difficulty and ability: Insights from computer-based large-scale assessment. Journal of Educational Psychology, 106, 608–626. https://doi.org/10.1037/a0034716
Goldhammer, F., Naumann, J., & Keßel, Y. (2013). Assessing individual differences in basic computer skills: Psychometric characteristics of an interactive performance measure. European Journal of Psychological Assessment, 29, 263–275. https://doi.org/10.1027/1015-5759/a000153
Hatlevik, O. E., Throndsen, I., Loi, M., & Gudmundsdottir, G. B. (2018). Students’ ICT self-efficacy and computer and information literacy: Determinants and relationships. Computers & Education, 118, 107–119. https://doi.org/10.1016/j.compedu.2017.11.011
Hatlevik, O. E., Scherer, R., & Christophersen, K.-A. (2017). Moving beyond the study of gender differences: An analysis of measurement invariance and differential item functioning of an ICT literacy scale. Computers & Education, 113, 280–293. https://doi.org/10.1016/j.compedu.2017.06.003
Hatlevik, O. E., Ottestad, G., & Throndsen, I. (2015). Predictors of digital competence in 7th grade: a multilevel analysis. Journal of Computer Assisted Learning, 31, 220–231. https://doi.org/10.1111/jcal.12065
Hatlevik, O. E., & Christophersen, K.-A. (2013). Digital competence at the beginning of upper secondary school: Identifying factors explaining digital inclusion. Computers & Education, 63, 240–247. https://doi.org/10.1016/j.compedu.2012.11.015
Hohlfeld, T. N., Ritzhaupt, A. D., & Barron, A. E. (2013). Are gender differences in perceived and demonstrated technology literacy significant? It depends on the model. Educational Technology Research and Development, 61(4), 639–663. https://doi.org/10.1007/s11423-013-9304-7
Jung, M., & Carstens, R. (Eds.). (2015). ICILS 2013 User Guide for the International Database. Amsterdam: IEA. https://doi.org/10.15478/uuid:73a9f018-7b64-4299-affc-dc33fe57f3e1
Livingstone, S., & Helsper, E. (2010). Balancing opportunities and risks in teenagers’ use of the internet: the role of online skills and internet self-efficacy. New Media & Society, 12(2), 309–329. https://doi.org/10.1177/1461444809342697
Luu, K., & Freeman, J. G. (2011). An analysis of the relationship between information and communication technology (ICT) and scientific literacy in Canada and Australia. Computers & Education, 56(4), 1072–1082. https://doi.org/10.1016/j.compedu.2010.11.008
Mirazchiyski, P. (2015). Analyzing ICILS 2013 data using the IEA IDB Analyzer. In M. Jung & R. Carstens (Eds.), ICILS 2013 User Guide for the International Database (pp. 49–86). Amsterdam: IEA. https://doi.org/10.15478/uuid:73a9f018-7b64-4299-affc-dc33fe57f3e1
Mohamad, I. B., & Usman, D. (2013). Standardization and its effects on k-means clustering algorithm. Research Journal of Applied Science, Engineering and Technology, 6(17), 3299–3303. https://doi.org/10.19026/rjaset.6.3638
Morris, D., & Trushell, J. (2014). Computer programming, ICT and gender in the classroom: a male-dominated domain or a female preserve? Research in teacher education, 4(1), 4–9. Retrieved from https://www.uel.ac.uk/wwwmedia/microsites/riste/Article-1-David-Morris-and-John-Trushell.pdf
Muthén, L. K. & Muthén, B. O. (2012). Mplus Statistical Analysis With Latent Variables. User’s Guide. Seventh Edition ed. Los Angeles, CA: Muthén & Muthén. Retrieved from https://www.statmodel.com/download/usersguide/Mplus%20Users%20Guide%20v6.pdf
Naumann, J. (2012). Belastungen und Ressourcen beim Lernen aus Text und Hypertext. [Costs and resources in learning from text and hypertext]. (Habilitation thesis). Goethe Universität Frankfurt, Frankfurt, Germany. https://doi.org/10.13140/RG.2.2.34203.46888
Naumann, J., & Goldhammer, F. (2017). Time-on-task effects in digital reading are non-linear and moderated by persons’ skills and tasks’ demands. Learning and Individual Differences, 53, 1–16. https://doi.org/10.1016/j.lindif.2016.10.002
Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A monte carlo simulation study. Structural Equation Modeling, 14(4), 535–569. https://doi.org/10.1080/10705510701575396
Oberski, D. (2016). Mixture Models: Latent Profile and Latent Class Analysis. In J. Robertson & M. Kaptein (Eds.), Modern Statistical Methods for HCI (pp. 275–287). Switzerland: Springer. https://doi.org/10.1007/978-3-319-26633-6
Pelgrum, W. J., Reinen, I. A. M. J. & Plomp, T. (1993). Schools, teachers, students and computers: A cross-national perspecitve. IEA-Comped Study Stage 2. Enschede: University of Twente. Retrieved from https://files.eric.ed.gov/fulltext/ED372734.pdf
Punter, R. A., Meelissen, M. R., & Glas, C. A. (2017). Gender differences in computer and information literacy: An exploration of the performances of girls and boys in ICILS 2013. European Educational Research Journal, 16(6), 762–780. https://doi.org/10.1177/1474904116672468
Rohatgi, A., Scherer, R., & Hatlevik, O. E. (2016). The role of ICT self-efficacy for students’ ICT use and their achievement in a computer and information literacy test. Computers & Education, 102, 103–116. https://doi.org/10.1016/j.compedu.2016.08.001
Stelter, A., Goldhammer, F., Naumann, J., & Rölke, H. (2015). Die Automatisierung prozeduralen Wissens: Eine Analyse basierend auf Prozessdaten [The automation of procedural knowledge: An analysis based on process data]. In J. Stiller & C. Laschke (Eds.), Berlin-Brandenburger Beitrage zur Bildungsforschung 2015: Herausforderungen, Befunde und Perspektiven Interdisziplinärer Bildungsforschung (pp. 111–132). Frankfurt am Main: Lang. https://doi.org/10.3726/978-3-653-04961-9
Su, S. (2017). Incorporating Response Times in Item Response Theory Models of Reading Comprehension Fluency. University of Minnesota Digital Conservancy. Retrieved from https://search.proquest.com/docview/2013525225?accountid=13049
Tein, J.-Y., Coxe, S., & Cham, H. (2013). Statistical power to detect the correct number of classes in latent profile analysis. Structural Equation Modeling, 20(4), 640–657. https://doi.org/10.1080/10705511.2013.824781
Thomson, S. (2015). Australian Students in a Digital World. Policy Insights, Issue 3. Melbourne: ACER. Retrieved from https://research.acer.edu.au/cgi/viewcontent.cgi?article=1002&context=policyinsights
Wang, S., Zhang, S., Douglas, J. & Culpepper, S. (2018). Using Response Times to Assess Learning Progress: A Joint Model for Responses and Response Times. Measurement: Interdisciplinary Research and Perspectives, 16(1), 45–58. https://doi.org/10.1080/15366367.2018.1435105
Zhong, Z.-J. (2011). From access to usage: The divide of self reported digital skills among adolescents. Computers and Education, 56(3), 736–746. https://doi.org/10.1016/j.compedu.2010.10.016
Zillien, N., & Hargittai, E. (2009). Digital distinction: Status-specific types of internet usage. Social Science Quarterly, 90(2), 274–291. https://doi.org/10.1111/j.1540-6237.2009.00617.x