Interconnection between constructs and consequences: a key validity consideration in K–12 English language proficiency assessments

Abedi, J. (2008). Classification system for English language learners: Issues and recommendations. Educational Measurement: Issues and Practice, 27(3), 17–31. https://doi.org/10.1111/j.1745-3992.2008.00125.x. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice. Oxford University Press. Bailey, A., Maher, C., & Wilkinson, L. C. (Eds.) (2018). Language, literacy, and learning in the STEM disciplines: Language counts for English learners. Routledge. Bailey, A. L. (2007). The language demands of school: Putting academic English to the test. Yale University Press. Bailey, A. L., & Huang, B. H. (2011). Do current English language development/proficiency standards reflect the English needed for success in school? Language Testing, 28(3), 343–365. https://doi.org/10.1177/0265532211404187. Bailey, A. L., & Wolf, M. K. (2020). The construct of English language proficiency in consideration of college and career ready standards. In M. K. Wolf (Ed.), Assessing English language proficiency in K–12 U.S. schools, (pp. 36–54). Routledge. https://doi.org/10.4324/9780429491689-3. Bennett, R. E. (2010). Cognitively based assessment of, for, and as learning: A preliminary theory of action for summative and formative assessment. Measurement: Interdisciplinary Research and Perspectives, 8, 70–91. https://doi.org/10.1080/15366367.2010.508686. Boals, T., Kenyon, D. M., Blair, A., Cranley, M. E., Wilmes, C., & Wright, L. J. (2015). Transformation in K–12 English language proficiency assessment changing contexts, changing constructs. Review of Research in Education, 39(1), 122–164. https://doi.org/10.3102/0091732X14556072. Bunch, G. C. (2013). Pedagogical language knowledge preparing mainstream teachers for English learners in the new standards era. Review of Research in Education, 37(1), 298–341. https://doi.org/10.3102/0091732X12461772. Butler, F. A., Bailey, A. L., Stevens, R., Huang, B., & Lord, C. (2004). Academic English in fifth-grade mathematics, science, and social studies textbooks (CSE technical report, 642). University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Callahan, R., & Shifrer, D. (2016). Equitable access for secondary English learner students course taking as evidence of EL program effectiveness. Educational Administration Quarterly, 52(3), 463–496. https://doi.org/10.1177/0013161X16648190. Chalhoub-Deville, M. (2016). Validity theory: reform policies, accountability testing, and consequences. Language Testing, 33(4), 453–472. https://doi.org/10.1177/0265532215593312. Chalhoub-Deville, M. (2020). Toward a model of validity in accountability testing. In M. K. Wolf (Ed.), Assessing English language proficiency in K–12 U.S. schools, (pp. 245–264). Routledge. Cheng, L., Watanabe, Y., & Curtis, A. (2004). Washback in language testing: Research contexts and methods. Lawrence Erlbaum Associates. Cook, H. G., Linquanti, R., Chinen, M., & Jung, H. (2012). National evaluation of Title III implementation supplemental report: Exploring approaches to setting English language proficiency performance criteria and monitoring English learner progress. U.S. Department of Education. Council of Chief State School Officers. (2014). English language proficiency (ELP) standards with correspondences to K–12 English language arts (ELA), mathematics, and science practices, K–12 ELA standards, and 6-12 literacy standards. Cummins, J. (2008). BICS and CALP: empirical and theoretical status of the distinction. In B. Street, & N. H. Hornberger (Eds.), Encyclopedia of language and education, (vol. 2, pp. 71–83). Springer. Deville, C., & Chalhoub-Deville, M. (2011). Accountability-assessment under No Child Left Behind: Agenda, practice, and future. Language Testing, 28(3), 307–321. https://doi.org/10.1177/0265532211400876. DiCerbo, P., Anstrom, K., Baker, L., & Rivera, C. (2014). A review of the literature on teaching academic English to English language learners. Review of Educational Research, 84(3), 446–482. https://doi.org/10.3102/0034654314532695. Every Student Succeeds Act. (2015). Public Law No. 114-354. Farnsworth, T. (2020). A review of validity evidence on K–12 English language proficiency assessment: Current state and future direction. In M. K. Wolf (Ed.), Assessing English language proficiency in K–12 U.S. schools, (pp. 75–91). Routledge. Forte, E., Kuti, L., & O’Day, J. (2012). National evaluation of Title III implementation: A survey of states’ English language proficiency standards. U.S. Department of Education. Retrieved from https://www2.ed.gov/rschstat/eval/title-iii/national-implementation-report.pdf. Gebhard, M., & Harman, R. (2011). Genre theory in K–12 schools: A response to high-stakes school reforms in the United States. Journal of Second Language Writing, 20(1), 45–55. https://doi.org/10.1016/j.jslw.2010.12.007. Hakuta, K., Butler, Y. G., & Witt, D. (2000). How long does it take English learners to attain proficiency? University of California Linguistic Minority Research Institute. Hakuta, K., & Pompa, D. (2017). Including English learners in your state Title I accountability plan. Council of Chief State School Officers. Retrieved from https://files.eric.ed.gov/fulltext/ED580945.pdf. Hakuta, K., Santos, M., & Fang, Z. (2013). Challenges and opportunities for language learning in the context of the CCSS and the NGSS. Journal of Adolescent & Adult Literacy, 56(6), 451–454. https://doi.org/10.1002/JAAL.164. Hamilton, L. S., Stecher, B. M., Marsh, J. A., McCombs, J. S., Robyn, A., Russel, J., Naftel, S., & Barney, H. (2007). Standards-based accountability under No Child Left Behind experiences of teachers and administrators in three states. Retrieved from https://www.rand.org/pubs/monographs/MG589.html. Hamilton, L. S., Stecher, B. M., & Yuan, K. (2012). Standards-based accountability in the United States: Lessons learned and future directions. Education Inquiry, 3(2), 149–170. https://doi.org/10.3402/edui.v3i2.22025. Haneda, M. (2014). From academic language to academic communication: Building on English learners’ resources. Linguistics and Education, 26, 126–135. https://doi.org/10.1016/j.linged.2014.01.004. Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000. Kim, A. A., Chapman, M., Kondo, A., & Wilmes, C. (2020). Examining the assessment literacy required for interpreting score reports: a focus on educators of K–12 English learners. Language Testing, 37(1), 54–75. https://doi.org/10.1177/0265532219859881. Lane, S. (2020). Test-based accountability systems: The importance of paying attention to consequences (research report no. RR-20-02). Educational Testing Service. https://doi.org/10.1002/ets2.12283. Lee, O. (2017). Common Core State Standards for ELA/literacy and next generation science standards: Convergences and discrepancies using argument as an example. Educational Researcher, 46(2), 90–102. https://doi.org/10.3102/0013189X17699172. Linn, R. L. (2000). Assessments and accountability. Educational Researcher, 29(2), 4–16. https://doi.org/10.2307/1177052. Linquanti, R., & Cook, H. G. (2015). Re-examining reclassification: Guidance from a national working session on policies and practices for exiting students from English learner status. Council of Chief State School Officers. Menken, K., Hudson, T., & Leung, C. (2014). Symposium: Language assessment in standards-based education reform. TESOL Quarterly, 48(3), 586–614. https://doi.org/10.1002/tesq.180. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement, (3rd ed., pp. 13–103). American Council on Education & Macmillan. Messick, S. (1996). Validity and washback in language testing. Language Testing, 13, 243–256. https://doi.org/10.1177/026553229601300302. Molle, D., & Wilfrid, J. (2021). Promoting multilingual students’ disciplinary and language learning through the WIDA framework for equitable instruction. Educational Researcher, 50(9), 585–594. https://doi.org/10.3102/0013189X211024592. Moschkovich, J. (2012). Mathematics, the Common Core, and language: Recommendations for mathematics instruction for ELs aligned with the Common Core, Commissioned paper by the Understanding Language Initiative (). Stanford University Retrieved from http://ell.stanford.edu/papers/. National Governors Association Center for Best Practices, Council of Chief State School Officers (2010). Common core state standards. National Research Council (2011a). Allocating federal funds for state programs for English language learners. National Academies Press. National Research Council (2011b). Incentives and test-based accountability in education. National Academies Press. Neugebauer, S. R., & Heineke, A. J. (2020). Unpacking K–12 teachers’ understandings of academic language. Teacher Educational Quarterly, 47(2), 158–182. NGSS Lead States (2013). Next generation science standards: For states, by states. National Academies Press. O’Day, J. A., & Smith, M. S. (2019). Opportunity for all: A framework for quality and equality in education. Harvard Education Press. Parker, C., Louie, J., & O’Dwyer, L. (2009). New measures of English language proficiency and their relationship to performance on large-scale content assessments. U.S. Department of Education. Retrieved from https://files.eric.ed.gov/fulltext/ED504060.pdf. Pereira, N., & de Oliveira, L. C. (2015). Meeting the linguistic needs of high-potential English language learners: What teachers need to know. Teaching Exceptional Children, 47(4), 208–215. https://doi.org/10.1177/0040059915569362. Porter, A., McMaken, J., Hwang, J., & Yang, R. (2011). Common Core Standards: The new U.S. intended curriculum. Educational Researcher, 40(3), 103–116. https://doi.org/10.3102/0013189X11405038. Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3–14. https://doi.org/10.3102/0013189X031007003. Sahlberg, P. (2006). Education reform for raising economic competitiveness. Journal of Educational Change, 7(4), 259–287. Schleppegrell, M. J. (2012). Academic language in teaching and learning: Introduction to special issue. The Elementary School Journal, 112(3), 409–418. Schrank, F. A., Fletcher, T. V., & Alvarado, C. G. (1996). Comparative validity of three English oral language proficiency tests. Bilingual Research Journal, 20(1), 55–68. https://doi.org/10.1080/15235882.1996.10668620. Snow, C. E. (2010). Academic language and the challenge of reading for learning. Science, 328(5977), 450–452. https://doi.org/10.1126/science.1182597. Solórzano, R. W. (2008). High stakes testing: Issues, implications, and remedies for English language learners. Review of Educational Research, 78(2), 260–329. https://doi.org/10.3102/0034654308317845. Spurrier, A., Alderman, C., Schiess, J., & Rotherham, A. J. (2020). The impact of standards-based accountability. Bellwether Education Partners Retrieved from https://files.eric.ed.gov/fulltext/ED606418.pdf. Stage, E. K., Asturias, H., Cheuk, T., Daro, P. A., & Hampton, S. B. (2013). Opportunities and challenges in next generation standards. Science, 340, 276–277. https://doi.org/10.1126/science.1234011. Tanenbaum, C., Boyle, A, Soga, K., Carlson, L., Golden, L., et al. (2012) National evaluation of Title III implementation: Report on state and local implementation. U.S. Department of Education. Retrieved from https://www2.ed.gov/rschstat/eval/title-iii/state-local-implementation-report.pdf. Tsagari, D., & Cheng, L. (2017). Washback, impact, and consequences revisited. In E. Shohamy, I. Or, & S. May (Eds.), Language testing and assessment, (3rd ed., pp. 359–372). Springer. U.S. Department of Education (2018). A state’s guide to the U.S. Department of Education’s assessment peer review process. Retrieved from https://www2.ed.gov/admins/lead/account/saa/assessmentpeerreview.pdf. U.S. Department of Education, Office of English Language Acquisition [OELA]. (2021). English learner population by local education agency fact sheet. Retrieved from https://ncela.ed.gov/files/fast_facts/20210315-FactSheet-ELPopulationbyLEA-508.pdf. Uccelli, P., Barr, C. D., Dobbs, C. L., Phillips Galloway, E., Meneses, A., & Sánchez, E. (2014). Core academic language skills (CALS): An expanded operational construct and a novel instrument to chart school—relevant language proficiency in pre-adolescent and adolescent learners. Applied Psycholinguistics, 36, 1077–1109. https://doi.org/10.1017/S014271641400006X. Umansky, I. M. (2016). Leveled and exclusionary tracking: English learners’ access to core content in middle school. American Educational Research Journal, 53(6), 1792–1833. https://doi.org/10.3102/0002831216675404. WIDA (2014). 2012 Amplification of the English language development standards, kindergarten–grade 12 (“WIDA ELD Standards”). Board of Regents of the University of Wisconsin System. WIDA (2020). WIDA English language development standards framework, 2020 edition, Kindergarten–grade 12. Board of Regents of the University of Wisconsin System. Wolf, M. K., Bailey, A. L., Ballard, L., Wang, Y., & Pogossian, A. (2022). Unpacking the language demands in academic content and English language proficiency standards for English learners. International Multilingual Research Journal. Advance online publication. https://doi.org/10.1080/19313152.2022.2116221. Wolf, M. K., & Farnsworth, T. (2014). English language proficiency assessments as an exit criterion for English learners. In A. Kunnan (Ed.), The companion to language assessment, (vol. 1, pp. 303–317). Wiley-Blackwell. https://doi.org/10.1002/9781118411360.wbcla118. Wolf, M. K., & Faulkner-Bond, M. (2016). Validating English language proficiency assessment uses for English learners: Academic language proficiency and content assessment performance. Educational Measurement: Issues and Practice, 35(2), 6–18. https://doi.org/10.1111/emip.12105. Wolf, M. K., Guzman-Orth, D., & Hauck, M. C. (2016). Next-generation summative English language proficiency assessments for English learners: Priorities for policy and research (ETS research report no. RR-16-08). Educational Testing Service. https://doi.org/10.1002/ets2.12091. Wolf, M. K., Kao, J., Herman, J. L., Bachman, L. F., Bailey, A., Bachman, P. L., … Chang, S. M. (2008). Issues in assessing English language learners: English language proficiency measures and accommodation uses - literature review (CRESST technical report 731). University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).