The influence of lexical features on teacher judgements of ESL argumentative essays

Alderson, 2014, Towards a theory of diagnosis in second and foreign language assessment: Insights from professional practice across diverse fields, Applied Linguistics, 36, 236, 10.1093/applin/amt046 Baumert, 2006, Stichwort: Professionelle Kompetenz von Lehrkräften, Zeitschrift für Erziehungswissenschaften, 4, 469, 10.1007/s11618-006-0165-2 Bearman, 2016, Support for assessment practice: Developing the assessment design decisions framework, Teaching in Higher Education, 21, 545, 10.1080/13562517.2016.1160217 Bechger, 2010, Detecting halo effects in performance-based examinations, Applied Psychological Measurement, 34, 607, 10.1177/0146621610367897 Bereiter, 1987 Birkel, 2002, How concordant are teachers’ essay scorings? A replication of Rudolf Weiss’ studies, Psychologie in Erziehung und Unterricht, 49, 219 Black, 1998, Assessment and classroom learning, Assessment in Education Principles Policy and Practice, 5, 7, 10.1080/0969595980050102 Black, 2003, ’In praise of educational research’: Formative assessment, British Educational Research Journal, 29, 623, 10.1080/0141192032000133721 Bloom, 1971 Board, 2010 Brunswik, 1956 Brupbacher, 2008, Englisch, 88 Coe, 2011 Council, 2001 Crossley, 2011, Understanding expert ratings of essay quality: Coh-Metrix analyses of first and second language writing, International Journal of Continuing Engineering Education and Life-Long Learning, 21, 170, 10.1504/IJCEELL.2011.040197 Crossley, 2012, Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication, Journal of Research in Reading, 35, 115, 10.1111/j.1467-9817.2010.01449.x Crossley, 2011, Predicting the proficiency level of language learners using lexical indices, Language Testing, 29, 243, 10.1177/0265532211419331 Crossley, 2013, Comparing count-based and band-based incides of word frequency: Implications for active vocabulary research and pedagogical applications, System, 41, 965, 10.1016/j.system.2013.08.002 Crossley, 2014, Linguistic microfeatures to predict L2 writing proficiency: A case study in automated writing Evaluation, Journal of Writing Assessment, 7 Crossley, 2013, Validating lexical measures using human scores of lexical proficiency, 105 Crossley, 2014, Assessing lexical proficiency using analytic ratings: A case for collocation accuracy, Applied Linguistics, 36, 570 Crusan, 2010 Crusan, 2016, Writing assessment literacy: Surveying second language teachers’ knowledge, beliefs, and practices, Assessing Writing, 28, 43, 10.1016/j.asw.2016.03.001 Culham, 2003 Cumming, 2005, Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL, Assessing Writing, 10, 5, 10.1016/j.asw.2005.02.001 Cumming, 2002, Decision making while rating ESL/EFL writing tasks: A descriptive framework, Modern Language Journal, 86, 67, 10.1111/1540-4781.00137 Daller, 2007 Durán, 2004, Developmental trends in lexical diversity, Applied Linguistics, 25, 220, 10.1093/applin/25.2.220 Eckes, 2008, Rater types in writing performance assessments: A classification approach to rater variability, Language Testing, 25, 155, 10.1177/0265532207086780 Eckes, 2015, Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments, Frankfurt am Main: Peter Lang. Eckes, 2016, Assessing writing, 147 EDK, 1994 EDK, 1995 Engber, 1995, The relationship of lexical proficiency to the quality of ESL compositions, Journal of Second Language Writing, 4, 139, 10.1016/1060-3743(95)90004-7 Engelhard, 2018 Federal Office for Statistics, B, 2014, vol. 15 Ferris, 1997, Teacher commentary on student writing: Descriptions & implications, Journal of Second Language Writing, 6, 155, 10.1016/S1060-3743(97)90032-1 Flower, 1981, A cognitive process theory of writing, National Council of Teachers of English, 32, 365 Graesser, 2004, Coh-Metrix: Analysis of text on cohesion and language, Behavior Research Methods Instruments & Computers, 36, 193, 10.3758/BF03195564 Grant, 2000, Using computer-tagged linguistic features to describe L2 writing differences, Journal of Second Language Writing, 9, 123, 10.1016/S1060-3743(00)00019-9 Guo, 2013, Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study, Assessing Writing, 18, 218, 10.1016/j.asw.2013.05.002 Hamp-Lyons, 2003, Writing teachers as assessors of writing, 162 Hamp-Lyons, 2016, Purposes of assessment, 13 Hawkey, 2004, Developing a common scale for the assessment of writing, Assessing Writing, 9, 122, 10.1016/j.asw.2004.06.001 Hyland, 1990, A genre description of the argumentative essay, RELC Journal, 66, 10.1177/003368829002100105 Hyland, 2008 Ingenkamp, 2008 Jarvis, 2002, Short texts, best fitting curves, and new measures of lexical diversity, Language Testing, 19, 57, 10.1191/0265532202lt220oa Jarvis, 2013, Defining and measuring lexical diversity, 13 Jarvis, 2013, Introduction, 1 Jarvis, 2003, Exploring multiple profiles of highly rated learner compositions, Journal of Second Language Writing, 12, 377, 10.1016/j.jslw.2003.09.001 Kaiser, 2015, Das Schülerinventar: Welche Schülermerkmale die Leistungsurteile von Lehrkräften beeinflussen, Zeitschrift für Erziehungswissenschaften, 18, 279, 10.1007/s11618-015-0619-5 Kaiser, 2016, The effects of student characteristics on teachers’ judgment accuracy: Disentangling ethnicity, minority status, and achievement, Journal of Educational Psychology, 109, 871, 10.1037/edu0000156 Keller, 2013 KMK, 2012 Knoch, 2009, Diagnostic assessment of writing: A comparison of two rating scales, Language Testing, 26, 275, 10.1177/0265532208101008 Knopp, 2012, Teilkomponenten von Schreibkompetenz untersuchen: Bericht aus einem interdisziplinären empirischen Projekt, 47 Kronig, 2007 Kyle, 2015, Automatically assessing lexical sophistication: Indices, tools, findings, and application, TESOL Quarterly, 49, 757, 10.1002/tesq.194 Kyle, 2016, The relationship between lexical sophistication and independent and source-based writing, Journal of Second Language Writing, 34, 12, 10.1016/j.jslw.2016.10.003 Laufer, 1995, Vocabulary size and use: Lexical richness in L2 written production, Applied Linguistics, 16, 307, 10.1093/applin/16.3.307 Lewis, 1993 Linnarud, 1986 Lu, 2012, The relationship of lexical richness to the quality of ESL learners’ oral narratives, Modern Language Journal, 96, 190, 10.1111/j.1540-4781.2011.01232_1.x Malvern, 2004 Marshall, 1967, Composition errors and essay examination grades re-examined, American Educational Research Journal, 4, 375, 10.3102/00028312004004375 McCarthy, 2005 McCarthy, 2007, Vocd: A theoretical and empirical evaluation, Language Testing, 24, 459, 10.1177/0265532207080767 McCarthy, 2010, MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment, Behavior Research Methods, 42, 381, 10.3758/BRM.42.2.381 McCarthy, 2013, From intrinsic to extrinsic issues of lexical diversity assessment - An ecological validation study, 45 McNamara, 2010, Linguistic features of writing quality, Written Communication, 27, 57, 10.1177/0741088309351547 McNamara, 2014 Meadows, 2010 Meier, 2016, Principled rubric adoption and adaptation: One multi-method case study, 165 Messick, 1994 Nation, 2011 Olinghouse, 2013, The relationship between vocabulary and writing quality in three genres, Reading and Writing, 26, 45, 10.1007/s11145-012-9392-5 Parr, 2010, Feedback to writing, assessment for teaching and learning and student progress, Assessing Writing, 15, 68, 10.1016/j.asw.2010.05.004 Porsch, 2010 Porsch, 2010, Standardbasiertes Testen von Schreibkompetenzen im Fach Englisch, 85 Rakedzon, 2017, To make a long story short: A rubric for assessing graduate students’ academic and popular science writing skills, Assessing Writing, 32, 28, 10.1016/j.asw.2016.12.004 Rauin, 2007, Subjektive Einschätzungen des Kompetenzerwerbs in der Lehramtsausbildung, 103 Read, 2000 Rezaei, 2010, Reliability and validity of rubrics for assessment through writing, Assessing Writing, 15, 18, 10.1016/j.asw.2010.01.003 Royal-Dawson, 2009, Is Teaching Experience Necessary for Reliable Scoring of Extended English Questions?, Educational Measurement Issues and Practice, 28, 2, 10.1111/j.1745-3992.2009.00142.x Sadler, 1989, Formative assessment and the design of instructional systems, Instructional Science, 18, 119, 10.1007/BF00117714 Scannell, 1966, The effect of selected composition errors on grades assigned to essay examinations, American Educational Research Journal, 3, 125, 10.3102/00028312003002125 Schmider, 2010, Is it really robust? Reinvestigating the robustness of ANOVA against violations of the normal distribution assumption, Methodology, 6, 147, 10.1027/1614-2241/a000016 Schrader, 2013, Diagnostische Kompetenz von Lehrpersonen, Beiträge zur Lehrerbildung, 31, 154, 10.36950/bzl.31.2013.9646 Scriven, 1967 Shohamy, 1992, The effect of raters’ background and training on the reliability of direct writing tests, Modern Language Journal, 76, 27, 10.1111/j.1540-4781.1992.tb02574.x Shrout, 1979, Intraclass correlations: Uses in assessing rater reliability, Psychological Bulletin, 86, 420, 10.1037/0033-2909.86.2.420 Shulman, 1987, Knowledge and teaching: Foundations of the new reform, Harvard Educational Review, 57, 1, 10.17763/haer.57.1.j463w79r56455411 Staples, 2016, Understanding first-year L2 writing: A lexico-grammatical analysis across L1s, genres, and language ratings, Journal of Second Language Writing, 32, 17, 10.1016/j.jslw.2016.02.002 Südkamp, 2012, Accuracy of teachers’ judgments of students academic achievement: A meta-analysis, Journal of Educational Psychology, 104, 743, 10.1037/a0027627 The British National Corpus, 2007 Thorndike, 1920, A constant error in psychological ratings, The Journal of Applied Psychology, 33, 263 Treffers-Daller, 2013, Measuring lexical diversity among L2 learners of French - An exploration of the validity of D, MTLD and HD-D as measures of language ability, 79 Weigle, 2002 Weigle, 2007, Teaching writing teachers about assessment, Journal of Second Language Writing, 16, 194, 10.1016/j.jslw.2007.07.004 Weir, 1988, The specification, realization and validation of an English language proficiency test, 45 White, 2009, Are you assessment literate? Some fundamental questions regarding effective classroom-based assessment, OnCUE Journal, 3, 3 Wind, 2017, Exploring the relationship between textual characteristics and rating quality in rater-mediated writing assessments: An illustration with L1 and L2 writing assessments, Assessing Writing, 34, 1, 10.1016/j.asw.2017.08.003 Wolfe, 2016, Features of difficult-to-score essays, Assessing Writing, 27, 1, 10.1016/j.asw.2015.06.002 Yu, 2010, Lexical diversity in writing and speaking task performances, Applied Linguistics, 31, 236, 10.1093/applin/amp024 Zemach, 2008 Zhu, 2001, Performing argumentative writing in english: Difficulties, processes, and strategies, TESL Canada Journal, 19, 34, 10.18806/tesl.v19i1.918