On the usefulness of CDA-based score reporting: implications for self-regulated learning

Yeonsook Yi1
1SangMyung University, Jongno-Gu, South Korea

Tóm tắt

AbstractIn cognitive diagnostic modeling research, one area that has not had enough research interests is remedial learning or instruction based on the information provided by cognitive diagnostic assessment (CDA). The present study tries to address this research gap by looking into the usefulness of the fine-grained score reports based on CDA in two different ways, i.e., a post-test and a survey inquiring about the perceived effectiveness of the score report that provided the skill profile of individual students. Another significance of the current research is that it attempted to introduce cognitive diagnostic assessment into a regular school exam unlike most previous studies that retrofitted to the existing tests. College students in Korea participated in the study, who were encouraged to do self-regulated learning utilizing the detailed information in the CDA-based score report. The results of the post-test and the survey were positive overall, supporting the utility of CDA-generated performance reports. The article ends with some suggestions for future research based on the limitations of the study.

Từ khóa


Tài liệu tham khảo

Baker, F. B. 2001. The basics of item response theory, ERIC Clearinghouse on Assessment and Evaluation, http://echo.edres.org:8080/irt/baker/

Bejar, I. I. (1984). Educational diagnostic assessment. Journal of Educational Measurement, 21, 175–189.

Brown, G., Irving, E., & Keegan, P. (2008). An introduction to educational assessment, measurement and evaluation. Pearson Prentice Hall.

Buck, G., & Tatsuoka, K. (1998). Application of the rule-space procedure to language testing: Examining attributes of a free response listening test. Language Testing, 15(2), 119–157.

Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: a theoretical synthesis. Review of Educational Research, 65, 245–281.

Cangelosi, J. S. (1990). Designing tests for evaluating student achievement. New York: Longman.

Ebel, R. L., & Frisbie, D. A. (1986). Essentials of educational measurement. Englewood Cliffs, NJ: Prentice Hall.

Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.

Jang, E. E. (2005). A validity narrative: Effects of reading skills diagnosis on teaching and learning in the context of NG TOEFL. In Unpublished doctoral dissertation. Urbana-Champaign, Urbana, IL: University of Illinois at.

Jang, E. E., Dunlop, M., Park, G., & van der Boom, E. (2015). How do young students with different profiles of reading skill mastery, perceived ability, and goal orientation respond to holistic diagnostic feedback? Language Testing, 32(3), 359–383.

Lee, Y.-W. (2015). Diagnosing diagnostic language assessment. Language Testing, 32(3), 299–316.

Li, H. (2011). Evaluating language group differences in the subskills of reading using a cognitive diagnostic modeling and differential skill functioning approach. University Park, PA: Unpublished doctoral dissertation, Pennsylvania State University.

Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide (Version 8) [Computer software and manual]. Los Angeles, CA: Muthén & Muthén.

Roberts, M. R., & Gierl, M. J. (2010). Developing score reports for cognitive diagnostic assessments. Educational Measurement: Issues and Practice, 29, 25–38.

Rupp, A., Templin, J., & Henson, R. (2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: The Guildford Press.

Sessoms, J., & Henson, R. A. (2018). Applications of diagnostic classification models: A literature review and critical commentary, measurement. Interdisciplinary Research and Perspectives, 16(1), 1–17.

Sun, Y., & Suzuki, M. (2013). Diagnostic assessment for improving teacher practice. International Journal of Information and Education Technology, 3, 607–610. https://doi.org/10.7763/IJIET.2013.V3.345.

van der Boom, E., & Jang, E. E. (2018). The effects of holistic diagnostic feedback intervention on improving struggling readers’ reading skills. The Journal of Teaching and Learning, 12(2), 54–69.

Wu, H.-M. (2018). Online individualized tutor for improving mathematics learning: A cognitive diagnostic model approach. Educational Psychology. https://doi.org/10.1080/01443410.2018.1494819.

Yi, Y.-S. (2017). In search of optimal cognitive diagnostic model(s) for ESL grammar test data. Applied Measurement in Education, 30(2), 82–101.