Differential impacts of e-portfolio assessment on language learners’ engagement modes and genre-based writing improvement
Tóm tắt
Portfolio assessment (PA) as an assessment for learning (AfL) alternative has been under-represented in second/foreign language acquisition (SLA) research literature. This study examined the potential impacts of electronic PA (e-PA) on English-as-a-Foreign-Language (EFL) learners’ engagement modes in descriptive and narrative genres of writing on Moodle™. To do so, 56 university students were non-randomly selected and assigned into two intermediate-level EFL cohorts. In a pretest-mediation-posttest study, descriptive and narrative writing tasks completed by two groups were subjected to teacher feedback, student reflection logs, and subsequent revision every week. Results of repeated measures ANOVA indicated significant progress in lower-level skills (sentence structure, word choice/grammar, mechanics), and moderate progress in higher-level skills (organization, development) in both groups’ genre-based writing. Results of one-way ANCOVA reported the notable pretest-to-posttest achievement by both groups with no intergroup statistical differences. The content of students’ reflection logs was inductively analyzed for their behavioral, emotional, and cognitive modes of engagement in e-PA. Qualitative data analysis indicated similar writing time intervals and recurrence of revisions as the behavioral mode of both groups. Participants also expressed novelty, low anxiety, and enjoyment as their emotional experiences. In terms of their cognitive experience, the majority agreed upon the applicability of teacher feedback and positive perception of writing improvement in e-PA. Yet, they were critical to regular mismatches between the scopes of teacher assessment and self-assessment, as well as teacher linguistic bias towards certain writing features. Several pedagogical implications of the study promote the facilitating role of e-PA in genre-based academic writing and e-learning contexts.
Tài liệu tham khảo
Afrianto, A. (2017). Challenges of using portfolio assessment as an alternative assessment method for teaching English in Indonesian schools. International Journal of Educational Best Practices, 1(2), 106–114. https://doi.org/10.31258/ijebp.v1n2.p106-114.
Ames, H., Glenton, C., & Lewin, S. (2019). Purposive sampling in a qualitative evidence synthesis: a worked example from a synthesis on parental perceptions of vaccination communication. BMC Medical Research Methodology, 19(26). https://doi.org/10.1186/s12874-019-0665-4.
Aydin, S. (2010). EFL writers’ perceptions of portfolio keeping. Assessing Writing, 15(3), 194–203. https://doi.org/10.1016/j.asw.2010.08.001.
Barrett, H. C. (2007). Researching electronic portfolios and learner engagement: The REFLECT initiative. Journal of Adolescent & Adult Literacy, 50(6), 436–449.
Barrot, J. S. (2021). Effects of Facebook-based e-portfolio on ESL learners' writing performance. Language, Culture and Curriculum, 34(1), 95–111. https://doi.org/10.1080/07908318.2020.1745822.
Baturay, M. H., & Daloğlu, A. (2010). E-portfolio assessment in an online English language course. Computer Assisted Language Learning, 23(5), 413–428. https://doi.org/10.1080/09588221.2010.520671.
Behbahani, S. M. K., Pourdana, N., Maleki, M., & Javanbakht, Z. (2011). EFL task-induced involvement and incidental vocabulary learning: succeeded or surrounded. In International Conference on Languages, Literature and Linguistics. IPEDR Proceedings, (vol. 26, pp. 323–325).
Benson, P. (2007). Autonomy and its role in learning. In J. Cummins, & C. Davison (Eds.), The international handbook of English language teaching, (vol. 2). Norwell, MA: Springer.
Borg, S. (2003). Teacher cognition in language teaching: A review of research on what language teachers think, know, believe, and do. Language Teaching, 36(2), 81–109. https://doi.org/10.1017/S0261444803001903.
Brown, H. D., & Abeywickrama, P. (2010). Language assessment: principles and classroom practices. White Plains, NY: Pearson Education.
Burner, T. (2014). The potential formative benefits of portfolio assessment in second and foreign language writing contexts: A review of the literature. Studies in Educational Evaluation, 43, 139–149.
Caner, A. (2010). Students views on using portfolio assessment an EFL writing courses. Anadolu University Journal of Social Sciences, 10(1), 223–236.
Carless, D. (2011). From testing to productive student learning: Implementing formative assessment in Confucian-heritage settings. New York: Routledge.
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354.
Chappuis, J. (2014). Seven strategies of assessment for learning, (2nd ed.). New York: Pearson.
Condon, W., & Hamp-Lyons, L. (1994). Maintaining a portfolio-based writing assessment: Research that informs program development. In L. Black, D. A. Daiker, J. Sommers, & G. Stygall (Eds.), New directions in portfolio assessment: Reflection practice, critical theory, and large-scale scoring, (pp. 277–285). Portsmouth, NH: Boynton/Cook.
Cowie, B. (2005). Pupil commentary on assessment for learning. Curriculum Journal, 16(2), 137–151. https://doi.org/10.1080/09585170500135921.
Darling-Hammond, L., & McCloskey, L. (2008). Assessment for learning around the world: What would it mean to be internationally competitive? The Phi Delta Kappan, 90(4), 263–272. https://doi.org/10.1177/003172170809000407.
De Fina, A., & Georgakopoulou, A. (2015). The handbook of narrative analysis. UK: Wiley and Blackwell.
Derewianka, B. (2003). Trends and issues in genre-based approaches. RELC Journal, 34(2), 133–154. https://doi.org/10.1177/003368820303400202.
Duff, P. (2010). Language socialization into academic discourse communities. Annual Review of Applied Linguistics, 30, 169–192. https://doi.org/10.1017/S0267190510000048.
Earl, L. M. (2013). Assessment as learning: Using classroom assessment to maximize student learning, (2nd ed., ). Thousand Oaks, CA: Corwin.
Ecclestone, K. (2007). Commitment, compliance and comfort zones: the effects of formative assessment on vocational education students’ learning careers. Assessment in Education: Principles. Policy & Practices, 14(3), 315–333. https://doi.org/10.1080/09695940701591925.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059.
Fredricks, J. A., & Eccles, J. S. (2002). Children’s competence and value beliefs from childhood through adolescence: Growth trajectories in two male-sex-typed domains. Developmental Psychology, 38(4), 519–533. https://doi.org/10.1037/0012-1649.38.4.519.
Gan, L., & Lam, R. (2020). Understanding university English instructors’ assessment training needs in the Chinese context. Language Testing in Asia, 10(11). https://doi.org/10.1186/s40468-020-00109-y.
Gearhart, M., Herman, J. L., Baker, E. L., & Whittaker, A. K. (1992). Writing portfolios at the elementary level: A study of methods for writing assessment (CSE Report 337). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
George, D., & Mallery, M. (2010). SPSS for Windows step by step: a simple guide and reference, 17.0 update, (10th ed., ). Boston: Pearson.
Gottlieb, M. (2000). Portfolio practices in elementary and secondary schools. In G. V. Ekbatani, & H. D. Pierson (Eds.), Learner-directed assessment in ESL, (pp. 89–104). Mahwah, NJ: Lawrence Erlbaum.
Gregory, K., Cameron, C., & Davies, A. (2001). Knowing what counts: conferencing and reporting. Merville, BC: Connections Publishing.
Halim, I. A., & Lestari, Z. W. (2019). the use of portfolio assessment of writing skill in descriptive text. Journal and Applied Linguistics and Literacy, 3(2), 75–85. https://doi.org/10.25157/jall.v3i2.2420.
Hamp-Lyons, L. (2007). The impact of testing practices on teaching: Ideologies and alternatives. In J. Cummins, & C. Davison (Eds.), International handbook of English language teaching, (pp. 487–504). Norwell, MA: Springer.
Hamp-Lyons, L. (2016). Purposes of assessment. In D. Tsagari, & J. Banerjee (Eds.), Handbook of second language assessment, (pp. 13–28). The Hague: De Gruyter/Mouton.
Hamp-Lyons, L., & Condon, W. (2000). Assessing the portfolio: Issues for research, theory and practice. Cresskill, NJ: Hampton Press.
Hargreaves, E. (2005). Assessment for learning? Thinking outside the (black) box. Cambridge Journal of Education, 35(2), 213–224. https://doi.org/10.1080/03057640500146880.
Hirvela, A., & Sweetland, Y. L. (2005). Two case studies of L2 writers’ experiences across learning directed portfolio contexts. Assessing Writing, 10(3), 192–213. https://doi.org/10.1016/j.asw.2005.07.001.
Hung, S. A. (2012). A washback study on e-portfolio assessment in an English as a Foreign Language teacher preparation program. Computer Assisted Language Learning, 25(1), 21–36. https://doi.org/10.1080/09588221.2010.551756.
Hyland, K. (2007). Genre pedagogy: Language, literacy and L2 writing instruction. Journal of Second Language Writing, 16(3), 148–164. https://doi.org/10.1016/j.jslw.2007.07.005.
Hyland, K. (2018). Genre and second language writing. In M. DelliCarpini (Ed.), The TESOL encyclopedia of English language teaching, (pp. 2359–2366). Wiley.
Hyland, K., & Hyland, F. (Eds.). (2019). Feedback in second language writing: Contexts and issues (pp. 1-22). Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/9781108635547.003
Ismailov, M., & Laurier, J. (2021). We are in the breakout room. Now what? An e-portfolio study of virtual team processes involving undergraduate online learners. E-Learning and Digital Media. https://doi.org/10.1177/20427530211039710.
Kirkpatrick, R., & Gyem, K. (2012). Washback effects of the new English assessment system on secondary schools in Bhutan. Language Testing in Asia, 29(5). https://doi.org/10.1186/2229-0443-2-4-5.
Klenowski, V. (2009). Assessment for learning revisited: an Asia-Pacific perspective. Assessment in Education: Principles, Policy, and Practice, 16(3), 263–268. https://doi.org/10.1080/09695940903319646.
Kusuma, I., Mahayanti, N. W. S., Adnyani, L. D. S., & Budiarta, L. G. R. (2021). Incorporating E-Portfolio with Flipped Classrooms: An In-Depth Analysis of Students' Speaking Performance and Learning Engagement. JALT CALL Journal, 17(2), 93–111. https://doi.org/10.29140/jaltcall.v17n2.378.
Lam, R. (2014). Promoting self-regulated learning through portfolio assessment: testimony and recommendations. Assessment & Evaluation in Higher Education, 39(6), 699-714.
Lam, R. (2018a). Promoting self-reflection in writing: a showcase portfolio approach. In A. Burns, & J. Siegel (Eds.), International perspectives on teaching skills in ELT, (pp. 219–231). London: Palgrave MacMillan.
Lam, R. (2018b). Teacher learning of portfolio assessment practices: testimonies of two writing teachers. In M. F. Hill & H. Jiang (Eds.), Teacher learning from classroom assessment: Perspectives from Asia Pacific (pp. 99–118). New York: Springer.
Lam, R. (2019). Writing portfolio assessment in practice: Individual, institutional, and systemic issues. Pedagogies: An International Journal, 15(3), 169–182. https://doi.org/10.1080/1554480X.2019.1696197.
Lam, R., & Lee, I. (2010). Balancing the dual functions of portfolio assessment. ELT Journal, 64(1), 54–64. https://doi.org/10.1093/elt/ccp024.
Lee, I. (2016). Teacher education on feedback in EFL writing: Issues, challenges, and future directions. TESOL Quarterly, 50(2), 518–527. https://doi.org/10.1002/tesq.303.
Lee, I., & Coniam, D. (2013). Introducing assessment for learning for EFL writing in an assessment of learning examination-driven system in Hong Kong. Journal of Second Language Writing, 22(1), 34–50. https://doi.org/10.1016/j.jslw.2012.11.003.
Lenhard, W., & Lenhard, A. (2016). Calculation of effect sizes. Bibergau: Psyhometrica.
Mphahlele, R. S. (2022). Digital assessment literacy in online courses (formative/summative): Rethinking assessment strategies in the open distance and e-learning institutions. In Handbook of research on managing and designing online courses in synchronous and asynchronous environments, (pp. 404–417). IGI Global.
NBCT (2015). Office of Assessment West Virginia Department of Education. Artificial intelligence scoring of student essays: West Virginia’s experience. Cameron Powers.
Ngui, W., Pang, V., Hiew, W., & Lee, K. W. (2020). Exploring the impact of e-portfolio on ESL students’ writing skills through the lenses of Malaysian undergraduates. Computer-Assisted Language Learning Electronic Journal, 21(3), 105–121.
O’Shea, A. (2020). Teachers’ conceptions of assessment for learning: what are the implications for children? Polish Journal of Educational Studies, 72(1), 64–78. https://doi.org/10.2478/poljes-2019-0005.
Panadero, E., & Romero, M. (2014). To rubric or not to rubric? The effects of self-assessment on self-regulation, performance, and self-efficacy. Assessment in Education Principles Policy and Practice, 21(2), 133–148. https://doi.org/10.1080/0969594X.2013.877872.
Pourdana, N., & Asghari, S. (2021). Different dimensions of teacher and peer assessment of EFL learners’ writing: descriptive and narrative genres in focus. Language Testing. Asia, 11(6). https://doi.org/10.1186/s40468-021-00122-9.
Pourdana, N., & Behbahani, S. M. K. (2013). Task types in EFL context: Accuracy, fluency, and complexity in assessing writing performance. International Journal of Social Science and Humanity, 2(1). https://doi.org/10.7763/IJSSH.2012.V2.73.
Price, M., Handley, K., Millar, J., & O’Donovan, B. (2010). Feedback: all that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277–289. https://doi.org/10.1080/02602930903541007.
Ramaprasad, A. (1983). On the definition of feedback. Systems Research and Behavioral Sciences, 28(1), 4–13. https://doi.org/10.1002/bs.3830280103.
Romova, Z., & Andrew, M. (2011). Teaching and assessing academic writing via the portfolio: Benefits for learners of English as an additional language. Assessing Writing, 16(2), 111–122. https://doi.org/10.1016/j.asw.2011.02.005.
Roohani, A., & Taheri, F. (2015). The effect of portfolio assessment on EFL learners’ expository writing ability. Iranian Journal of Language Testing, 5(1), 45–59.
Saadatmandi, M., Modarres, K. S., & Pourdana, N. (2018). Teaching English pragmatic features in EFL context: A focus on request speech acts. Theory and Practice in Language Studies, 8(7), 829–835. https://doi.org/10.17507/tpls.0807.14.
Sadler, D. R. (2010). Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535–550. https://doi.org/10.1080/02602930903541015.
Sandiford, C., & Macken-Horarik, M. (2020). Changing stories: linguistically-informed assessment of development in narrative writing. Assessing writing, 45, 1–12. https://doi.org/10.1016/j.asw.2020.100471.
Shober, L. S. (1996). A portfolio assessment approach to narrative writing with the cooperation of a fourth grade, ERIC, ED395318.
Song, B., & August, B. (2002). Using portfolios to assess the writing of ESL students: a powerful alternative? Journal of Second Language Writing, 11(1), 49–72. https://doi.org/10.1016/S1060-3743(02)00053-X.
Steen-Utheim, A., & Hopfenbeck, T. N. (2018). To do or not to do with feedback: A study of undergraduate students’ engagement and use of feedback within a portfolio assessment design. Assessment & Evaluation in Higher Education, 44(1), 80–96. https://doi.org/10.1080/02602938.2018.1476669.
Tesch, R. (1990). Qualitative research: Analysis types and software tools. New York: Falmer.
Vygotsky, L. S. (1987). Thinking and speech. In R. W. Rieber, & A. S. Carton (Eds.), The collected works of L. S. Vygotsky: Vol. 1: Problems of general psychology, (pp. 39–285). Plenum.
Willis, J. (2011). Affiliation, autonomy and assessment for learning. Assessment in Education: Principles, Policy & Practice, 18(4), 399–415. https://doi.org/10.1080/0969594X.2011.604305.
Xu, Y., & Brown, G. T. L. (2016). Teacher assessment literacy in practice: A reconceptualization. Teaching and Teacher Education, 58, 149–162. https://doi.org/10.1016/j.tate.2016.05.010.
Yancey, K. B. (1998). Reflection in the writing classroom. USU Press Publications.
Yu, S. (2020). Giving genre-based peer feedback in academic writing: sources of knowledge and skills, difficulties and challenges. Assessment & Evaluation in Higher Education, 46(1), 36–53. https://doi.org/10.1080/02602938.2020.1742872.
Zhang, Z. V., & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36, 90–102. https://doi.org/10.1016/j.asw.2018.02.004.