Effect of Observation Mode on Measures of Secondary Mathematics Teaching
Tóm tắt
Classroom observation of teachers is a significant part of educational measurement; measurements of teacher practice are being used in teacher evaluation systems across the country. This research investigated whether observations made live in the classroom and from video recording of the same lessons yielded similar inferences about teaching. Using scores on the Classroom Assessment Scoring System–Secondary (CLASS-S) from 82 algebra classrooms, we explored the effect of observation mode on inferences about the level or ranking of teaching in a single lesson or in a classroom for a year. We estimated the correlation between scores from the two observation modes and tested for mode differences in the distribution of scores, the sources of variance in scores, and the reliability of scores using generalizability and decision studies for the latter comparisons. Inferences about teaching in a classroom for a year were relatively insensitive to observation mode. However, time trends in the raters’ use of the score scale were significant for two CLASS-S domains, leading to mode differences in the reliability and inferences drawn from individual lessons. Implications for different modes of classroom observation with the CLASS-S are discussed.
Từ khóa
Tài liệu tham khảo
Bill and Melinda Gates Foundation. (2012, January). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains. Seattle, WA: Author. Retrieved from http://www.metproject.org/downloads/MET_Gathering_Feedback_Research_Paper.pdf
Brunvard S., 2010, Contemporary Issues in Technology and Teacher Education, 10, 247
Erickson F., 2006, Handbook of complementary methods in education research, 177
Frederiksen J. R., 1992, Video portfolio assessment: A study for the National Board for Professional Teaching Standards
Gitomer D. H., Bell C. A., Qi Y., McCaffrey D. F., Hamre B. K., Pianta R. C. (in press). The instructional challenge in improving teaching quality: Lessons from a classroom observation protocol. Teachers College Record.
Hastie T. J., 1990, Generalized additive models
Jaeger R. M. (1993, April). Live vs. Memorex: Psychometric and practical issues in the collection of data on teachers’ performances in the classroom. Paper presented at the meeting of the American Educational Research Association, Atlanta, GA. (ERIC Document Reproduction Service No. ED360325). Retrieved from http://www.eric.ed.gov/PDFS/ED360325.pdf
Mashburn A. J., Downer J. T., Rivers S. E., Brackett M. A., Martinez A. (2011). Improving the power of an experimental study of a social and emotional learning program: Application of generalizability theory to the measurement of classroom-level outcomes. Unpublished manuscript.
Miller K., 2007, Video research in the learning sciences, 321
Pianta R. C., 2007, Classroom assessment scoring system manual, middle/secondary version
Pianta R. C., 2008, Classroom assessment scoring system, manual, K-3
Shavelson R. J., 1991, Generalizability theory: A primer