Organization of ideas in writing: what are raters sensitive to?
Tóm tắt
Whether foreign language writing is rated using analytic rating scales or holistically, the organization of ideas is invariably one of the aspects assessed. However, it is unclear what raters are sensitive to when rating writing for organization. Which do they value more highly, the physical aspects of organization, such as paragraphing and the existence of organization markers, or deeper textual aspects, such as the coherent flow of ideas? This study investigates whether raters of timed essays value paragraphing, cohesive devices or coherence more when assigning a score for organization. The current study used multiple regression to ascertain what raters of the writing section of an in-house proficiency test were sensitive to when rating writing for organization using an analytic rating scale. The number of paragraphs, number of cohesive devices and coherence within 116 timed essays were evaluated and it was found that raters seem to value the physical aspects of organization more than the deeper textual aspects. Specifically, the number of paragraphs and the number of cohesive devices predicted differences in scores assigned for organization. In addition, the scores assigned for content were significantly predictive of scores for organization.
Tài liệu tham khảo
Bachman L, Palmer A: Language assessment in practice. Oxford: Oxford University Press; 2010.
Bamberg B: What makes a text coherent? Coll Compos Commun 1983, 34(4):417–429. 10.2307/357898
Bamberg B: Assessing coherence: A reanalysis of essays written for the National Assessment of Educational Progress, 1969–1979. Res Teach Engl 1984, 18(3):305–319.
Carrell P, L,: Cohesion is not coherence. TESOL Q 1982, 16(4):479–488. 10.2307/3586466
Carson J: Reading for writing: Cognitive perspectives. In Reading in the composition classroom: Second language perspectives. Edited by: Carson J, Leki I. Boston: Heinle & Heinle; 1993:85–104.
Charney D: The validity of using holistic scoring to evaluate writing: A critical overview. Res Teach Engl 1984, 18(1):65–81.
Cohen J: A power primer. Psychol Bull 1992, 112(1):155–159.
Connor U: A study of cohesion and coherence in English as a second language students’ writing. Res Lang Soc Interact 1984, 17(3):301–316.
Connor U: Linguistic/rhetorical measures for international persuasive student writing. Res Teach Engl 1990, 24: 67–87.
Connor U, Farmer M: The teaching of topical structure analysis as a revision strategy for ESL writers. In Second language writing: Research insights for the classroom. Edited by: Kroll B. Cambridge: Cambridge University Press; 1991.
Cooper C: Holistic evaluation in writing. In Evaluating writing. Edited by: Cooper C, Odell L. National Council of Teachers of English: Urbana, IL; 1977.
Crismore A, Markannen R, Steffensen M: Metadiscourse in persuasive writing: A study of texts written by American and Finnish university students. Writ Commun 1993, 10(1):39–71. 10.1177/0741088393010001002
Cummings A, Kantor R, Powers D: Scoring TOEFL essays and TOEFL 2000 prototype writing tasks: An investigation into raters’ decision making, and development of a preliminary analytic framework. TOEFL Monograph Series. Princeton, NJ: Educational Testing Service; 2001.
Erdosy M, U,: Exploring variability in judging writing ability in a second language: A study of four experienced raters of ESL compositions. (TOEFL Research Report No. RR-70). Princeton, NJ: ETS; 2004.
Ferris D: Does error feedback help student writers? In Feedback in second language writing: Contexts and issues. Edited by: Hyland K, Hyland F. New York: Cambridge University Press; 2006:89.
Folse K, S,, Solomon E, V,, Clabeaux D: From great paragraphs to great essays. Boston, MA: Heinle Cengage Learning; 2010.
Freedman S: How characteristics of student essays influence teachers’ evaluations. J Educ Psychol 1979, 71(3):328–338.
Grabe W, Kaplan R, B,: Theory and practice of writing. London: Longman; 1996.
Harris W: Teacher response to student writing: A study of the response patterns of high school English teachers to determine the basis for teacher judgment of student writing. Res Teach Engl 1977, 11: 175–185.
Hinds J: Reader versus writer responsibility: A new typology. In Writing across languages: Analysis of L2 text. Edited by: Connor U, Kaplan RB. Reading, MA: Addison-Wesley; 1987:141–152.
Hirose K: Product and process in the L1 and L2 writing of Japanese students of English. Hiroshima: Keisuisha; 2005.
Hyland K: Disciplinary interactions: Metadiscourse in L2 postgraduate writing. J Second Lang Writ 2004, 13: 133–151. 10.1016/j.jslw.2004.02.001
Hyland K, Tse P: Metadiscourse in academic writing. Applied Linguistics 2004, 25(2):156–177. 10.1093/applin/25.2.156
IELTS. (n.d.): An overview of IELTS academic writing. Retrieved from http://www.ielts.org/pdf/Writing%20Band%20descriptors%20Task%202.pdf
Jacobs H, L,, Zinkgraf S, A,, Wormuth D, R,, Hartfiel V, F,, Hughey J, B,: Testing ESL composition: A practical approach. Rowley, MA: Newbury House; 1981.
Johns A: Coherence and academic writing: Some definitions and suggestions for teaching. TESOL Quarterly 1986, 20(2):247–265. 10.2307/3586543
Lee I: Teaching coherence to ESL students: A classroom inquiry. J Second Lang Writ 2002, 11: 135–159. 10.1016/S1060-3743(02)00065-6
Lumley T: Assessment criteria in a large-scale writing test: What do they really mean to the raters? Lang Test 2002, 19(3):246–276. 10.1191/0265532202lt230oa
Lumley T: Assessing second language writing: A rater’s perspective. Frankfurt: Peter Lang Publishing; 2005.
Knoch U: ‘Little coherence, considerable strain for reader’: A comparison between two rating scales for the assessment of coherence. Assess Writ 2007, 12: 108–128. 10.1016/j.asw.2007.07.002
Kobayashi H, Rinnert C: Effects of first language on second language writing: Translation versus direct composition. Lang Learn 1992, 42(2):183–215. 10.1111/j.1467-1770.1992.tb00707.x
MacIntyre R: Revision of a criterion-referenced rating scale used to assess academic writing. Studies in Linguistics and Language Teaching 2007, 18: 203–219.
Morgan J, L,, Sellner M, B,: Discourse and linguistic theory. In Theoretical issues in reading comprehension. Edited by: Spiro RJ, Bertram BC, Brewer WF. Hillsdale, NJ: Lawrence Erlbaum Associates; 1980.
Raimes A: Grammar troublespots: A guide for writers. New York: Cambridge University Press; 2004.
SAT. (n.d.): Essay scoring: How it’s scored and what the scores mean. Retrieved from http://sat.collegeboard.org/scores/sat-essay-scoring-guide
Vande Kopple W: Some exploratory discourse on metadiscourse. Coll Compos Commun 1985, 36(1):82–93. 10.2307/357609
van Dijk T, A,: Macrostructures: An interdisciplinary study of global structures in discourse, interaction, and cognition. Hillsdale, NJ: Erlbaum; 1980.
Vaughan C: Holistic assessment: What goes on in the rater’s mind? In Assessing Second Language Writing in Academic Contexts. Edited by: Hamp-Lyons L. Norwood, NJ: Ablex; 1991:111–125.
Weigle S: Assessing Writing. Cambridge: Cambridge University Press; 2002.
Whithaus C, Harrison S, Midyette J: Keyboarding compared to handwriting on a high stakes writing assessment: Student choice of composing medium, raters’ perceptions and text quality. Assess Writ 2008, 13(1):4–25. 10.1016/j.asw.2008.03.001
Williams J: Style: Ten lessons in clarity and grace. Glenview: Scott Forseman; 1981.
Witte S, P,, Faigley L,: Coherence, cohesion, and writing quality. Coll Compos Commun 1981, 32(2):189–204. 10.2307/356693
Zhang P, Li N: An assessment of human-computer interaction research in management information systems: Topics and methods. Computers in Human Behaviour 2004, 20: 125–147. 10.1016/j.chb.2003.10.011