Linguistic complexity in teachers' assessment of German essays in high stakes testing

Alexopoulou, 2017, Task effects on linguistic complexity and accuracy: A large-scale learner corpus analysis employing natural language processing techniques, Language Learning, 67, 180, 10.1111/lang.12232 Baayen, 2008, Mixed-effects modeling with crossed random effects for subjects and items, Journal of Memory and Language, 59, 390, 10.1016/j.jml.2007.12.005 2007 Bates, 2015, Fitting linear mixed-effects models using lme4, Journal of Statistical Software, 67, 1, 10.18637/jss.v067.i01 Beers, 2009, Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre?, Reading and Writing, 22, 185, 10.1007/s11145-007-9107-5 Beers, 2011, Writing development in four genres from grades three to seven: Syntactic complexity and genre differentiation, Reading and Writing, 24, 183, 10.1007/s11145-010-9264-9 Berendes, 2018, Reading demands in secondary school: Does the linguistic complexity of textbooks increase with grade level and the academic orientation of the school track?, Journal of Educational Psychology, 110, 518, 10.1037/edu0000225 Berkling, K., Fay, J., Ghayoomi, M., Hein, K., Lavalley, R., Linhuber, L., Stüker, S. (May 2014). A database of freely written texts of German school students for the purpose of automatic spelling error classification. In: The 9th edition of the language resources and evaluation conference (LREC 2014).Reykjavik, Iceland, 1212–1217. Biber, 2016, Predicting patterns of grammatical complexity across language exam task types and proficiency levels, Applied Linguistics, 37, 639, 10.1093/applin/amu059 Brezina, 2019, Morphological complexity in written L2 texts, Second Language Research, 35, 99, 10.1177/0267658316643125 Brysbaert, 2011, The word frequency effect: A review of recent developments and implications for the choice of frequency estimates in German, Experimental Psychology, 58, 412, 10.1027/1618-3169/a000123 Bulté, 2014, Conceptualizing and measuring short-term changes in L2 writing complexity, Journal of Second Language Writing, 26, 42, 10.1016/j.jslw.2014.09.005 Chen, X., Meurers, D. (2016). CTAP: A web-based tool supporting automatic complexity analysis. In: Proceedings of the workshop on computational linguistics for linguistic complexity. COLING, Osaka, Japan, 113–119. Cohen, 1992, A power primer, Psychological Bulletin, 112, 155, 10.1037/0033-2909.112.1.155 Crossley, 2020, Linguistic features in writing quality and development: An overview, Journal of Writing Research, 11, 415, 10.17239/jowr-2020.11.03.01 Crossley, 2014, Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners, Journal of Second Language Writing, 26, 66, 10.1016/j.jslw.2014.09.006 Crossley, 2011, Predicting human scores of essay quality using computational indices of linguistic and textual features, 438 Cumming, 2002, Decision making while rating ESL/EFL writing tasks: A descriptive framework, The Modern Language Journal, 86, 67, 10.1111/1540-4781.00137 Ellis, 2003 Gibson, 2000, The dependency locality theory: A distance-based theory of linguistic complexity, Image, Language, Brain, 95 Hancke, J., Vajjala, S., Meurers, D. (2012). Readability classification for German using lexical, syntactic, and morphological features. In: Proceedings of the 24th international conference on computational linguistics (COLING). Mumbay, India, 1063–1080, http://aclweb.org/anthology-new/C/C12/C12-1065.pdf. Heister, 2011, dlexDB - eine lexikalische Datenbank für die psychologische und linguistische Forschung [dlexDB - A lexical database for the psychological and linguistic research], Psychologische Rundschau, 62, 191 Hennig, 2013, Unpersönliches Schreiben in der Wissenschaft [Impersonal writing in science], Informationen Deutsch als Fremdsprache, 4, 439, 10.1515/infodaf-2013-0407 Housen, 2019, Multiple approaches to complexity in second language research, Second Language Research Special Issue on Linguistic Complexity, 35, 2 Housen, 2009, Complexity, accuracy, and fluency in second language acquisition, Applied Linguistics, 30, 461, 10.1093/applin/amp048 Housen, 2012, Complexity, accuracy and fluency: Definitions, measurement and research, 1, 10.1075/lllt.32.01hou Hunt, 1965 Huot, 1990, The literature of direct writing assessment: Major concerns and prevailing trends, Review of Educational Research, 60, 237, 10.3102/00346543060002237 KMK (2014a). Bildungsstandards für die fortgeführte Fremdsprache (Englisch/Französisch) für die Allgemeine Hochschulreife [Educational standards for upper secondary education in the subjects English and French]. Sekretariat der Ständigen Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland (KMK). Köln: Wolters Kluwer. KMK (2014b). Bildungsstandards im Fach Deutsch für die Allgemeine Hochschulreife [Educational standards for upper secondary education in the subject German]. Sekretariat der Ständigen Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland (KMK).Köln: Wolters Kluwer. KMK (2017). The education system in the Federal Republic of Germany 2015/2016. A description of the responsibilities, structures and developments in education policy for the exchange of information in Europe. Sekretariat der Ständigen Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland (KMK). Retrieved from 〈https://www.kmk.org/fileadmin/Dateien/pdf/Eurydice/Bildungswesen-engl-pdfs/dossier_en_ebook.pdf〉. KMK (2018). Vereinbarung zur Gestaltung der gymnasialen Oberstufe und der Abiturprüfung. Beschluss der Kultusministerkonferenz vom 07.07.1972 i.d.F. vom 15.02.2018. Sekretariat der Ständigen Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland (KMK). Köln: Wolters Kluwer. Kühberger, 2019, Task complexity in history textbooks. A multidisciplinary case study on triangulation in history education research, History Education International Research Journal, 16 Laufer, 1995, Vocabulary size and use: lexical richness in L2 written production, Applied Linguistics, 16, 307, 10.1093/applin/16.3.307 McNamara, 2010, Linguistic features of writing quality, Written Communication, 27, 57, 10.1177/0741088309351547 Messick, 1994 Morek, M., Heller, V. (2012). Bildungssprache - kommunikative, epistemische, soziale und interaktive Aspekte ihres Gebrauchs. Zeitschrift für angewandte Linguistik, 67–101. Retrieved from 10.1515/zfal-2012–0011. Michel, 2019, Effects of task type on morphosyntactic complexity across proficiency: Evidence from a large learner corpus of A1 to C2 writings, Instructed Second Language Acquisition, 3, 124, 10.1558/isla.38248 Neuner, 1987, Cohesive ties and chains in good and poor freshman essays, Research in the Teaching of English, 21, 92 NGACBP & CCSSO(2010). Common core state standards for english language arts & literacy in history/social studies, science, and technical subjects. Appendix A. Washington, DC: National Governors Association Center for Best Practices (NGACBP), Council of Chief State School Officers (CCSSO). Norris, 2009, Towards an organic approach to investigating CAF in instructed SLA: The case of complexity, Applied Linguistics, 30, 555, 10.1093/applin/amp044 Okinina, N., Frey, J.-C., Weiss, Z. (2020). CTAP for Italian: Integrating components for the analysis of Italian into a multilingual linguistic complexity analysis tool. In: Proceedings of the 12th language resources and evaluation conference (LREC 2020). Pallotti, 2009, CAF: Defining, refining and differentiating constructs, Applied Linguistics, 30, 590, 10.1093/applin/amp045 Paquot, 2019, The phraseological dimension in interlanguage complexity research, Second Language Research, 35, 121, 10.1177/0267658317694221 Pilán, I., Vajjala, S., Volodina, E. (2015). A readable read: Automatic assessment of language learning materials based on linguistic complexity. In: Proceedings of CICLING 2015- Research in Computing Science Journal Issue (to appear). https://arxiv.org/abs/1603.08868. 2018 Rezaei, 2010, Reliability and validity of rubrics for assessment through writing, Assessing Writing, 15, 18, 10.1016/j.asw.2010.01.003 Robinson, 1995, Task complexity and second language narrative discourse, Learning Language, 45, 99, 10.1111/j.1467-1770.1995.tb00964.x Sadler, 1989, Formative assessment and the design of instructional systems, Instructional Science, 18, 119, 10.1007/BF00117714 Schleppegrell, 2001, Linguistic features of the language of schooling, Linguistics and Education, 12, 431, 10.1016/S0898-5898(01)00073-0 Schleppegrell, M.J. (2004). The Language of Schooling. A funtional Linguistics Perspective. Lawrence Erlbaum Associates Assosiates Publishers, Mahwah, New Jersey and London. Skehan, 1998 Thorndike, 1920, A constant error in psychological ratings, Journal of Applied Psychology, 33, 263 Vögelin, 2019, The influence of lexical features on teacher judgements of ESL argumentative essays, Assessing Writing, 39, 50, 10.1016/j.asw.2018.12.003 Weiss, Z., Meurers, D. (August 2018). Modeling the readability of German targeting adults and children: An empirically broad analysis and its cross-corpus validation. In: Proceedings of the 27th international conference on computational linguistics (COLING). International Committee on Computational Linguistic, Santa Fe, New Mexico, USA. Weiss, Z., Meurers, D. (Aug. 2019a). Analyzing linguistic complexity and accuracy in academic language development of German across elementary and secondary school.In: Proceedings of the fourteenth workshop on innovative use of NLP for building educational applications.association for computational linguistics, Florence, Italy, 380–393. https://www.aclweb.org/anthology/W19-4440. Weiss, Z., Meurers, D. (2019b). Broad linguistic modeling is beneficial for German L2 proficiency assessment. In: Abel, A.Glaznieks, A., Lyding, V., Nicolas, L., (Eds.), Widening the scope of learner corpus research. Selected Papers from the Fourth Learner Corpus Research Conference. Presses Universitaires de Louvain, Louvain-La-Neuve. Weiss, 2019, Computationally modeling the impact of task-appropriate language complexity and accuracy on human grading of German essays, Proceedings of the fourteenth workshop on innovative use of NLP for building educational applications. Florence, Italy: Association for Computational Linguistics, 30, 10.18653/v1/W19-4404 Wolfe, 2016, Features of difficult-to-score essays, Assessing Writing, 27, 1, 10.1016/j.asw.2015.06.002 Yoon, 2017, Linguistic complexity in L2 writing revisited: Issues of topic, proficiency, and construct multidimensionality, System, 66, 130, 10.1016/j.system.2017.03.007 Yoon, 2016, The linguistic development of students of English as a second language in two written genres, TESOL Quarterly, 51, 275, 10.1002/tesq.296