International assessment of low reading proficiency in the adult population: A question of components or lower rungs?
Tóm tắt
Among the United Nations’ 17 Sustainable Development Goals (SDGs) launched in 2015, the fourth goal (SDG 4) is dedicated to education, and one of the ten targets within that goal specifically addresses adult literacy and numeracy skills. Efforts to reach this target involve monitoring, which in turn involves assessment. The most powerful instrument for assessing literacy proficiency is the Programme for the International Assessment of Adult Competencies (PIAAC), conducted by the Organisation for Economic Co-operation and Development (OECD). It has five hierarchically organised proficiency levels for literacy. A sixth category, labelled “below Level 1”, lumps together low proficiencies at the bottom end of the proficiency continuum. To boost effective action in addressing SDG 4, the UNESCO Institute for Statistics (UIS) recently launched the Global Alliance to Monitor Learning (GAML), which aims to support national assessment strategies and to develop internationally comparable indicators and methodological measurement tools. While PIAAC Levels 1–5 are already broadly suitable for international comparison, the “below Level 1” category has so far only been assessed by individual countries (e.g. Canada, the United States, the United Kingdom and Germany) using instruments developed nationally. Focusing on the reading aspect of literacy, the authors of this article investigate how these nationally developed low proficiency assessment instruments might be adjusted to facilitate international comparability.
Tài liệu tham khảo
Addey, C. (2018). Assembling literacy as global: The danger of a single story. In M. Milana, J. Holford, S. Webb, P. Jarvis, & R. Waller (Eds.), The Palgrave international handbook of adult and lifelong education and learning (pp. 315–335). London: Palgrave Macmillan UK.
Bamberger, R., & Vanecek, E. (1984). Lesen – Verstehen – Lernen – Schreiben. Die Schwierigkeitsstufen von Texten in deutscher Sprache [Reading – understanding – learning – writing: The difficulty levels of texts in German]. Frankfurt am Main: Diesterweg.
BIS (Department for Business Innovation and Skills) (2012). The 2011 Skills for Life Survey: A survey of literacy, numeracy and ICT levels in England. London: BIS. Retrieved 21 February 2020 from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/36000/12-p168-2011-skills-for-life-survey.pdf.
Brooks, G., Giles, K., Harman, J., Kendall, S., Rees, F., & Whittaker, S. (2001). Assembling the fragments. A review of research on adult basic skills. Research report no. 220. London: Department for Education and Employment (DfEE).
Brooks, G., Davies, R., Duckett, L., Hutchinson, D., Kendall, S., & Wilkin, A. (2001). Progress in adult literacy. Do learners learn?. London: Basic Skills Agency.
Brügelmann, H. (2000). Kinder auf dem Weg zur Schrift. Eine Fibel für Lehrer und Laien. [Children on the way to writing: A primer for teachers and laypeople]. Bottighofen: Libelle.
Carlson, J. E., & von Davier, M. (2013). Item response theory. ETS R&D Scientific and Policy Contributions Series, ETS SPC-13-05. Princeton, NJ: Educational Testing Service (ETS). Retrieved 6 February 2020 from https://www.ets.org/Media/Research/pdf/RR-13-28.pdf.
Coltheart, M., Rastle, K., Perry, C., Langdon, R., & Ziegler, J. C. (2001). DRC: A dual route cascaded model of visual word recognition and reading aloud. Psychological Review, 108(1), 204–256.
de Ayala, R. J. (2009). The theory and practice of item response theory. New York: The Guilford Press.
Denckla, M. B., & Cutting, L. E. (1999). History and significance of rapid automatized naming. Annals of Dyslexia, 49(1), 29–42.
Dessinger, Y. (2011). Kompetenzmodelle des Schriftspracherwerbs [Competence models of written language acquisition]. In A. Grotlüschen, R. Kretschmann, E. Quante-Brandt, & K. D. Wolf (Eds.), Literalitätsentwicklung von Arbeitskräften [Literacy development of the labour force] (pp. 68–85). Münster: Waxmann.
Dunn, L. M. (1959). Peabody picture vocabulary test. Circle Pines, MS: American Guidance Service.
DVV (Deutscher Volkshochschul-Verband) (2014). Rahmencurriculum und Kurskonzept für die abschlussorientierte Grundbildung [Framework curriculum and course concept for graduation-oriented basic education]. Bonn: DVV. Retrieved 13 July 2016 from https://grundbildung.de/projekte/rahmencurriculum/material.html.
Elfert, M. (2019). Lifelong learning in Sustainable Development Goal 4: What does it mean for UNESCO’s rights-based approach to adult learning and education? International Review of Education, 65(4), 537–556.
Embretson, S. E., & Reise, S. (2000). Item response theory for psychologists. Psychometric methods. Mahwah, NJ: Lawrence Erlbaum Associates.
Frith, U. (1985). Beneath the surface of developmental dyslexia. In K. Patterson, J.C. Marshall, & M. Coltheart (Eds), Surface dyslexia. Neuropsychological and cognitive studies of phonological reading (pp. 301–330). London: Lawrence Erlbaum. Retrieved 2 August 2016 from https://www.icn.ucl.ac.uk/dev_group/ufrith/documents/Frith,%20Beneath%20the%20surface%20of%20developmental%20dyslexia%20copy.pdf.
Grek, S. (2019). METRO – International organisations and the rise of a global metrological field - ERC. Impact, 2019(1), 41–43.
Grotlüschen, A. (2011). Zur Auflösung von Mythen. Eine theoretische Verortung des Forschungsansatzes lea. – Literalitätsentwicklung von Arbeitskräften [Dispelling myths: A theoretical location of the lea. research approach. – Literacy development of the labour force]. In A. Grotlüschen, R. Kretschmann, E. Quante-Brandt & K. D. Wolf (Eds), Literalitätsentwicklung von Arbeitskräften [Literacy development of the labour force] (pp. 12–39). Münster: Waxmann.
Grotlüschen, A. (2018). Global competence: Does the new OECD competence domain ignore the Global South? Studies in the Education of Adult, 50(2), 185–202.
Grotlüschen, A., Dessinger, Y., Heinemann, A.M.B., & Schepers, C. (2010). Alpha-Levels Schreiben [Alpha levels writing]. lea.-Verlinkungsstudie Blog [blogpost July]. Hamburg: Hamburg University. Retrieved 2 August 2016 from https://blogs.epb.uni-hamburg.de/lea/files/2009/09/Kompetenzmodell-Schreiben.pdf.
Grotlüschen, A., Riekmann, W., & Buddeberg, K. (2012). Leo. – Level-One Studie: Methodische Herausforderungen [Leo. – Level-one study: Methodological challenges]. In A. Grotlüschen, & W. Riekmann (Eds), Funktionaler Analphabetismus in Deutschland. Ergebnisse der ersten leo. – Level-One Studie [Functional illiteracy in Germany: Results of the first Leo. – Level-one study] (pp. 54–76). Münster: Waxmann.
Guadalupe, C., & Cardoso, M. (2011). Measuring the continuum of literacy skills among adults: Educational testing and the LAMP experience. International Review of Education, 57(1–2), 199–217.
Hamilton, M., Maddox, B., & Addey, C. (Eds.). (2015). Literacy as numbers: Researching the politics and practices of international literacy assessment. The Cambridge Research series. Cambridge: Cambridge University Press.
Kelava, A., & Moosbrugger, H. (2012). Deskriptivstatistische Evaluation von Items (Itemanalyse) und Testwertverteilungen [Descriptive statistical evaluation of items (item analysis) and test value distributions]. In H. Moosbrugger & A. Kelava (Eds.), Testtheorie und Fragebogenkonstruktion [Testing theory and questionnaire construction] (pp. 75–102). Berlin/Heidelberg: Springer.
Kirsch, I., & Thorn, W. (2013). Foreword. The Programme for International Assessment of Adult Compentencies: An overview. In: OECD (Ed.), Technical report of the survey of adult skills (PIAAC) (pp. 1–20). Paris: OECD Publishing. Pre-publication copy retrieved 7 February 2020 from https://www.oecd.org/skills/piaac/_Technical%20Report_17OCT13.pdf.
Kretschmann, R. (2005). Prozessdiagnose der Schriftsprachkompetenz in den Schuljahren 1 und 2 [Process diagnosis of written language competence in primary Grades 1 and 2]. Bergedorfer Förderprogramme. Horneburg: Persen.
Kretschmann, R. (2011). Kompetenzverfahren „Leseverständnis“ (KLV) [Competence procedure “reading comprehension” (KLV)]. In A. Grotlüschen, R. Kretschmann, E. Quante-Brandt, & K. D. Wolf (Eds.), Literalitätsentwicklung von Arbeitskräften [Literacy development of the labour force] (pp. 41–57). Münster: Waxmann.
Lunz, M. (2010). Using the very useful Wright map [dedicated web page]. Chicago, IL: Measurement Research Associates, Inc. Retrieved 27 February 2020 from https://www.rasch.org/mra/mra-01-10.htm.
Moosbrugger, H. (2012). Item-response-theory (IRT). In H. Moosbrugger & A. Kelava (Eds.), Testtheorie und Fragebogenkonstruktion [Testing theory and questionnaire construction] (pp. 227–274). Berlin/Heidelberg: Springer.
Murray, S. (2001). Understanding the skills of low-literate adults: A proposal. Ottawa, ON: Statistics Canada.
Murray, T. S., Jones, S., Willms, D., Shillington, R., McCracken, M., & Glickman, V. (2008). Reading the future: Planning to meet Canada’s future literacy needs. Ottawa, ON: Canadian Council on Learning.
OECD (Organisation for Economic Co-Operation and Development). (2013). OECD Skills outlook 2013: First results from the survey of adult skills. OECD Publishing. https://doi.org/10.1787/9789264204256-en.
OECD & Statistics Canada. (2000). Literacy in the information age. Final report of the International Adult Literacy Survey. Paris: OECD.
OECD & Statistics Canada. (2005). Learning a living: First results of the Adult Literacy and Life Skills Survey. Ottawa/Paris: Statistics Canada/OECD. Retrieved 6 February 2020 from https://www150.statcan.gc.ca/n1/en/pub/89-603-x/2005001/pdf/4200878-eng.pdf?st=0qyXmqEp.
Ordinate Corporation. (1999). PhonePass testing: Structure and construct. Menlo Park, CA: Ordinate.
QCA (Qualifications and Curriculum Authority) (2005). National standards for adult literacy, numeracy and ICT. London: QCA. Retrieved 7 February 2020 from https://set.et-foundation.co.uk/media/131246/2005_national_standards_for_adult_literacy_numeracy_ict.pdf.
Rammstedt, B., Zabal, A., Martin, S., Perry, A., Helmschrott, S., Massing, N., Ackermann, D., & Maehler, D. (2015). ZA5845: Programme for the International Assessment of Adult Competencies (PIAAC), Germany – Reduced version. ZA5845 Data file Version 2.0.0. Cologne: GESIS Data Archive, Cologne. https://doi.org/10.4232/1.12385 [restricted access].
Reuter-Liehr, C. (2008). Eine Einführung in das Training der phonemischen Strategie auf der Basis des rhythmischen Syllabierens mit einer Darstellung des Übergangs zur morphemischen Strategie [An introduction to phonemic strategy training on the basis of rhythmic syllabi with a presentation of the transition to morphic strategy]. Bochum: Winkler.
Rost, J. (2004). Lehrbuch Testtheorie – Testkonstruktion [Textbook of testing theory and test construction]. Berne: Huber.
Sabatini, J.P. (2015). Understanding the basic reading skills of U.S. adults:. Reading components in the PIAAC literacy survey. Princeton, NJ: Educational Testing Service (ETS). Retrieved 7 February 2020 from https://www.ets.org/s/research/report/reading-skills/ets-adult-reading-skills-2015.pdf.
Sabatini, J.P., & Bruce, K.M. (2009). PIAAC reading components: A conceptual framework. OECD Education Working Papers, no. 33. Paris: OECD Publishing. https://doi.org/10.1787/220367414132
Schermelleh-Engel, K., & Werner, C. S. (2012). Methoden der Reliabilitätsbestimmung [Methods of reliability determination]. In H. Moosbrugger & A. Kelava (Eds.), Testtheorie und Fragebogenkonstruktion [Testing theory and questionnaire construction] (pp. 119–141). Berlin/Heidelberg: Springer.
Spitta, G. (1997). Kinder schreiben eigene Texte: Klasse 1 und 2. Lesen und Schreiben im Zusammenhang; spontanes Schreiben; Schreibprojekte [Children write their own texts: Grades 1 and 2. Reading and writing in context; spontaneous writing; writing projects]. Frankfurt am Main: Cornelsen.
Strucker, J., & Davidson, R. (2003). Adult Reading Components Study (ARCS). Harvard, MA: National Center for the Study of Adult Learning and Literacy (NCSALL). Retrieved 2 August 2016 from https://www.ncsall.net/fileadmin/resources/teach/prac_res_guide_read2.pdf.
Strucker, J., Yamamoto, K., & Kirsch, I. (2007). The relationship of the component skills of reading to performance on the International Adult Literacy Survey (IALS). Harvard, MA: National Center for the Study of Adult Learning and Literacy (NCSALL). Retrieved 2 August 2016 from https://www.ncsall.net/fileadmin/resources/research/reading_ials_rb.pdf.
Tamassia, C., Lennon, M.L., & Yamamoto, K. (2013). Scoring reliability studies. In: OECD (Ed.), Technical report of the survey of adult skills (PIAAC) (chapter 12). Pre-Publication Copy retrieved 7 February 2020 from https://www.oecd.org/skills/piaac/_Technical%20Report_17OCT13.pdf.
Torgesen, J. K., Wagner, R., & Rashotte, C. (1999). TOWRE: Test of Word Reading Efficiency. Austin, TX: Pro-Ed.
UIL (UNESCO Institute for Lifelong Learning) (2019). Progress on indicator 4.6.1. GAML 6 Meeting 27–28 August, Yerevan, Armenia [presentation slides]. Hamburg: UIL. Retrieved 11 February 2020 from https://gaml.uis.unesco.org/wp-content/uploads/sites/2/2019/05/TF4.6.GAML6_UILPresentation_clean.pdf.
UIS (United Nations Institute for Statistics) (2017). Global Alliance to Monitor Learning [dedicated webpage]. Retrieved 12 February 2020 from https://gaml.uis.unesco.org/.
UN (United Nations) (2015). Transforming our world: The 2030 Agenda for sustainable development. A/RES/70/1. New York: UN. Retrieved 11 February 2020 from https://sustainabledevelopment.un.org/post2015/transformingourworld/publication.
UN (2016). Sustainable Development Goal 4: Targets and indicators [dedicated webpage]. New York: UN. Retrieved 11 February 2020 from https://sustainabledevelopment.un.org/sdg4#targets.
Wechsler, D. (1997). WMS-III: Wechsler Memory Scale administration and scoring manual. New York: Psychological Corporation.
Wilson, M. (2005). Constructing measures. An item response modeling approach. Mahwah, NJ: Lawrence Erlbaum Associates.
Wu, M. L., Adams, R. J., & Wilson, M. R. (2007). ACER ConQuest version 2.0. Generalised item response modelling software. Camberwell, VIC: ACER Press.
Yamamoto, K., Khorramdel, L., & von Davier, M. (2013). Scaling PIAAC cognitive data. In OECD (Ed.), Technical report of the survey of adult skills (PIAAC) (chapter 17). Paris: OECD Publishing. Pre-publication copy retrieved 7 February 2020 from https://www.oecd.org/skills/piaac/_Technical%20Report_17OCT13.pdf.
Zabal, A., Martin, S., Massing, N., Ackermann, D., Helmschrott, S., Barkow, I., et al., & Rammstedt, B. (Eds.). (2014). PIAAC Germany 2012: Technical report. Münster: Waxmann.