The validity of student ratings of teaching quality: Factorial structure, comparability, and the relation to achievement

AERA, APA, NCME Standards for Educational and Psychological Testing 2014 American Educational Research Association. Aleamoni, 1999, Student rating myths versus research facts from 1924 to 1998, Journal of Personnel Evaluation in Education, 13, 153, 10.1023/A:1008168421283 Alp Christ, 2022, Learning processes and their mediating role between teaching quality and student achievement: A systematic review, Studies in Educational Evaluation, 75, 10.1016/j.stueduc.2022.101209 Atlay, 2019, The role of socioeconomic background and prior achievement for students’ perception of teacher support, British Journal of Sociology of Education, 40, 970, 10.1080/01425692.2019.1642737 Baumert, 2010, Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress, American Educational Research Journal, 47, 133, 10.3102/0002831209345157 Bellens, 2019, Teaching quality: catalyst or pitfall in educational systems’ aim for high achievement and equity? An answer based on multilevel SEM analyses of TIMSS 2015 data in Flanders (Belgium), Germany, and Norway, Large-Scale Assessments in Education, 7, 1, 10.1186/s40536-019-0069-2 Benton, 2012, Student ratings of teaching: A summary of esearch and literature, IEA, Paper, 50, 1 Bijlsma, 2022, Factors related to differences in digitally measured student perceptions of teaching quality, School Effectiveness and School Improvement, 33, 360, 10.1080/09243453.2021.2023584 Bliese, 2000, Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis, 349 Blikstad-Balas, 2021, Why – and How – Should We Measure Instructional Quality?, 9 Blömeke, 2019, Consistency of results regarding teacher effects across subjects, school levels, outcomes and countries, Teaching and Teacher Education, 77, 170, 10.1016/j.tate.2018.09.018 Brophy, 1983, Classroom Organization and Management, The Elementary School Journal, 83, 265, 10.1086/461318 Brophy, 1999 Brown, 2015 Can, 2015, Collinear latent variables in multilevel confirmatory factor analysis: A comparison of maximum likelihood and bayesian estimations, Educational and psychological Measurement, 75, 406, 10.1177/0013164414547959 Coe, 2014, “What makes great teaching?” Review of the underpinning research, Project Report, Sutton Trust Charalambous, 2021, Working more collaboratively to better understand teaching and its quality: challenges faced and possible solutions, Studies in Educational Evaluation, 71, 10109, 10.1016/j.stueduc.2021.101092 Cheung, 2002, Evaluating goodness-of-fit indexes for testing measurement invariance, Structural Equation Modeling: A Multidisciplinary Journal, 9, 233, 10.1207/S15328007SEM0902_5 Cotter, 2020, Developing the TIMSS 2019 mathematics and science achievement instruments, 1.1 Creemers, 2008 Danielson, 2007, Enhancing Professional Practice: A Framework for Teaching, 2nd edition, Association for Supervision and Curriculum Development De Jong, 2001, The quality of student ratings of teacher behaviour, Learning Environments Research, 4, 51, 10.1023/A:1011402608575 Desa, 2019, Measurement invariance in international large-scale assessments: Integrating theory and method, 879 Doyle, 1985, Recent research on classroom management: Implications for teacher education, Journal of Teacher Education, 36, 31, 10.1177/002248718503600307 Downer, 2015, Measuring effective teacher-student interactions from a student perspective: A multi-level analysis, The Journal of Early Adolescence, 35, 722, 10.1177/0272431614564059 Emmer, 2001, Classroom management: A critical part of educational psychology, with implications for teacher education, Educational Psychologist, 36, 103, 10.1207/S15326985EP3602_5 Eriksson, 2019, Using TIMSS items to evaluate the effectiveness of different instructional practices, An International Journal of the Learning Sciences, 47, 1 Fauth, 2014, Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes, Learning and Instruction, 29, 1, 10.1016/j.learninstruc.2013.07.001 Fauth, 2020, Who sees what?: conceptual considerations on the measurement of teaching quality from different perspectives, Zeitschrift für Pädagogik, 66, 138 Fauth, 2020, Don’t blame the teacher? The need to account for classroom characteristics in evaluations of teaching quality, Journal of Educational Psychology, 112, 1284, 10.1037/edu0000416 Fernández-García, 2019, Student perceptions of secondary education teaching effectiveness: General profile, the role of personal factors, and educational level, Frontiers in Psychology, 10, 10.3389/fpsyg.2019.00533 Geldhof, 2014, Reliability estimation in a multilevel confirmatory factor analysis framework, Psychological Methods, 19, 72, 10.1037/a0032138 Göllner, 2021, Student Ratings of Teaching Quality Dimensions: Empirical Findings and Future Directions, 111 He, 2022, Cross-Cultural Comparability of Latent Constructs in ILSAs Herbert, 2022, How valid are student perceptions of teaching quality across education systems, Learning and Instruction, 82, 10.1016/j.learninstruc.2022.101652 Hu, 1999, Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives, Structural Equation Modeling, 6, 1, 10.1080/10705519909540118 Hsu, 2015, Detecting misspecified multilevel structural equation models with common fit indices: A Monte Carlo study, Multivariate Behavioral Research, 50, 197, 10.1080/00273171.2014.977429 Corp, 2019, IBM SPSS Statistics for Windows Jaekel, 2022, “The teacher motivates us – or me?” – The role of the addressee in student ratings of teacher support, Contemporary Educational Psychology, 71, 10.1016/j.cedpsych.2022.102120 Klieme, 2013, The role of large-scale assessments in research on educational effectiveness and school development, 115 Klieme, 2022, Teaching Quality and Student Outcomes in TIMSS and PISA Klieme, 2009, The Pythagoras Study. Investigating effects of teaching and learning in Swiss and German mathematics classrooms, 137 Kounin, 1970, Discipline and group management in classrooms Kunter, 2008, Students’ and mathematics teachers’ perceptions of teacher enthusiasm and instruction, Learning and Instruction, 18, 468, 10.1016/j.learninstruc.2008.06.008 Lai, 2021, Composite reliability of multilevel data: It’s about observed scores and construct meanings, Psychological Methods, 26, 90, 10.1037/met0000287 LaRoche, 2020, Sample design in TIMSS 2019, 3.1 Laukaityte, 2018, Importance of sampling weights in multilevel modeling of international large-scale assessment data, Communications in Statistics - Theory and Methods, 47, 4991, 10.1080/03610926.2017.1383429 Lebreton, 2008, Answers to 20 questions about interrater reliability and interrater agreement, Organizational Research Methods, 11, 815, 10.1177/1094428106296642 Lipowsky, 2009, Quality of geometry instruction and its short-term impact on students’ understanding of the Pythagorean Theorem, Learning and Instruction, 19, 527, 10.1016/j.learninstruc.2008.11.001 Lüdtke, 2011, A 2 × 2 taxonomy of multilevel latent contextual models: Accuracy–bias trade-offs in full and partial error correction models, Psychological Methods, 16, 444, 10.1037/a0024376 Lüdtke, 2009, Assessing the impact of learning environments: How to use student ratings of classroom or school characteristics in multilevel modeling, Contemporary Educational Psychology, 34, 120, 10.1016/j.cedpsych.2008.12.001 Lüdtke, 2006, Reliability and agreement of student ratings of the classroom environment: A reanalysis of TIMSS data, An International Journal, 9, 215 Marder, 2021, Ask me, I (Dis)agree! Acquiescence in student ratings of teaching quality in German vocational schools, Studies in Educational Evaluation, 68, 10.1016/j.stueduc.2020.100937 Marsh, 2019, A tale of two quests: The (almost) non-overlapping research literatures on students’ evaluations of secondary-school and university teachers, Contemporary Educational Psychology, 58, 1, 10.1016/j.cedpsych.2019.01.011 Marsh, 2004, In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings, Structural Equation Modeling, 11, 320, 10.1207/s15328007sem1103_2 Marsh, 2012, Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects, Educational Psychologist, 47, 106, 10.1080/00461520.2012.670488 Maulana, 2016, Observations and student perceptions of the quality of preservice teachers’ teaching behaviour: construct representation and predictive quality, Learning Environments Research, 19, 335, 10.1007/s10984-016-9215-8 Millsap, 2011 Morin, 2014, Doubly latent multilevel analyses of classroom climate: An illustration, The Journal of Experimental Education, 82, 143, 10.1080/00220973.2013.769412 Mullis, I.V.S., & Martin, M.O. (2017). TIMSS 2019 Assessment Frameworks. Retrieved from Boston College, TIMSS & PIRLS International Study Center website: http://timssandpirls.bc.edu/timss2019/frameworks/. Muthén, 1998 Nilsen, 2016, Conceptual Framework and Methodology of This Report, 1 Pianta, 2009, Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity, Educational Researcher, 38, 109, 10.3102/0013189X09332374 Pianta, 2008 Praetorius, 2018, Classroom observation frameworks for studying teaching quality: Looking back and looking forward, ZDM, 50, 535, 10.1007/s11858-018-0946-0 Praetorius, 2018, Generic dimensions of teaching quality: The German framework of Three Basic Dimensions, ZDM, 50, 407, 10.1007/s11858-018-0918-4 Praetorius, 2020, Towards developing a theory of generic teaching quality: Origin, current status, and necessary next steps regarding the three basic dimensions model, Zeitschrift für Pädagogik, 66, 15 Putnick, 2016, Measurement invariance conventions and reporting: The state of the art and future directions for psychological research, Developmental Review, 41, 71, 10.1016/j.dr.2016.06.004 Reeve, 2012, A Self-determination Theory Perspective on Student Engagement, 149 Rollett, 2021, Student Feedback on Teaching in Schools: Current State of Research and Future Perspectives, 259 Röhl, 2021, The Process Model of Student Feedback on Teaching (SFT): A Theoretical Framework and Introductory Remarks, 1 Ryan, 2000, Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being, American Psychologist, 55, 68, 10.1037/0003-066X.55.1.68 Scheerens, 2007, Review and meta-analyses of school and teaching effectiveness, Universiteit Twente, Afdeling Onderwijsorganisatie Enōsis Management Scherer, 2015, Student assessment of teaching as a source of information about aspects of teaching quality in multiple subject domains: An application of multilevel bifactor structural equation modeling, Frontiers in Psychology, 6 Scherer, 2016, The Relations Among School Climate, Instructional Quality, and Achievement Motivation in Mathematics, 51 Schlesinger, 2018, Subject-specific characteristics of teaching quality in mathematics education, ZDM, 50, 475, 10.1007/s11858-018-0917-5 Schweig, 2016, Moving beyond means: Revealing features of the learning environment by investigating the consensus among student ratings, Learning Environments Research, 19, 441, 10.1007/s10984-016-9216-7 Seidel, 2007, Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results, Review of Educational Research, 77, 454, 10.3102/0034654307310317 Senden, 2021, Instructional Quality: A Review of Conceptualizations, Measurement Approaches, and Research Findings, 140 Stapleton, 2016, Construct Meaning in Multilevel Settings, Journal of Educational and Behavioral Statistics, 41, 481, 10.3102/1076998616646200 Teig, 2019, I know i can, but do i have the time? The role of teachers’ self-efficacy and perceived time constraints in implementing cognitive-activation strategies in science, Frontiers in Psychology, 10, 10.3389/fpsyg.2019.01697 van de Schoot, 2012, A checklist for testing measurement invariance, European Journal of Developmental Psychology, 9, 486, 10.1080/17405629.2012.686740 van der Scheer, 2019, Validity and reliability of student perceptions of teaching quality in primary education, School Effectiveness and School Improvement, 30, 30, 10.1080/09243453.2018.1539015 Von Davier, 2009, What are plausible values and why are they useful, Vol. 2, 9 Wagner, 2013, Construct validity of student perceptions of teaching quality is high, but not perfect: Dimensionality and generalizability of domain-independent assessments, Learning and Instruction, 28, 1, 10.1016/j.learninstruc.2013.03.003 Wisniewski, 2020, Obtaining secondary students’ perceptions of teaching quality: Two-level structure and measurement invariance, Learning and Instruction, 66