Effect size measures for multilevel models: definition, interpretation, and TIMSS example

Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park: Sage.

Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01 .

BIFIE. (2017). BIFIEsurvey: Tools for survey statistics in educational assessment. R package version 1.13-24. https://CRAN.R-project.org/package=BIFIEsurvey . Accessed 29 Jan 2018.

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.

Darlington, R. B., & Hayes, A. F. (2017). Regression analysis and linear models: Concepts, applications, and implementation. New York: Guilford Press.

Denson, N., & Seltzer, M. H. (2011). Meta-analysis in higher education: An illustrative example using hierarchical linear modeling. Research in Higher Education, 52, 215–244.

Ferron, J. M., Hogarty, K. Y., Dedrick, R. F., Hess, M. R., Niles, J. D., & Kromrey, J. D. (2008). Reporting results from multilevel analyses. In A. A. O’Connell & D. B. McCoach (Eds.), Multilevel modeling of educational data (pp. 391–426). Charlotte: Information Age Publishing Inc.

Hedges, L. V., & Hedberg, E. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60–87.

Hox, J. J. (2010). Multilevel analysis: Techniques and applications. Great Britain: Routledge.

Kelley, K., & Preacher, K. J. (2012). On effect size. Psychological Methods, 17, 137–152. https://doi.org/10.1037/a0028086 .

Kirk, R. E. (2013). Experimental design: Procedures for the behavioral sciences. Los Angeles: Sage.

Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modelling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1(3), 86–92. https://doi.org/10.1027/1614-1881.1.3.86 .

Martin, M. O., & Mullis, I. V. S. (Eds.). (2012). Methods and procedures in TIMSS and PIRLS 2011. Chestnut Hill: TIMSS & PIRLS International Study Center, Boston College.

McCoach, D. B., & Black, A. C. (2008). Evaluation of model fit and adequacy. In A. A. O’Connell & D. B. McCoach (Eds.), Multilevel modeling of educational data (pp. 245–272). Charlotte: Information Age Publishing Inc.

Meinck, S. & Vandenplas, C. (2012). Sample size requirements in HLM: An empirical study. IERI monograph series issues and methodologies in large-scale assessments. IER institute, special issue 1, educational testing service and international association for the evaluation of educational achievement.

Peng, C. Y. J., & Chen, L. T. (2013). Beyond Cohen’s d: Alternative effect size measures for between-subject designs. Journal of Experimental Education, 82, 22–50. https://doi.org/10.1080/00220973.2012.745471 .

R Core Team. (2014). R: A language and environment for statistical computing (Computer software), R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/ . Accessed 29 Jan 2018.

Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163.

Rogers, A. M., & Stoeckel, J. J. (2008). NAEP 2008 arts: Music and visual arts restricted-use data files data companion (NCES 2011–470). Washington, D.C: US Department of Education Institute of Education Sciences, National Center for Education Statistics.

Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Thousand Oaks: Sage Publishing.

Spybrook, J. (2008). Power, sample size, and design. In A. A. O’Connell & D. B. McCoach (Eds.), Multilevel modeling of educational data (pp. 273–311). Charlotte: Information Age Publishing Inc.

Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Boston: Pearson.

Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31, 114–128.