When Rater Reliability Is Not Enough
Tóm tắt
In recent years, interest has grown in using classroom observation as a means to several ends, including teacher development, teacher evaluation, and impact evaluation of classroom-based interventions. Although education practitioners and researchers have developed numerous observational instruments for these purposes, many developers fail to specify important criteria regarding instrument use. In this article, the authors argue that for classroom observation to succeed in its aims, improved observational systems must be developed. These systems should include not only observational instruments but also scoring designs capable of producing reliable and cost-efficient scores and processes for rater recruitment, training, and certification. To illustrate how such a system might be developed and improved, the authors provide an empirical example that applies generalizability theory to data from a mathematics observational instrument.
Từ khóa
Tài liệu tham khảo
Boston Public Schools. (2010). Evaluation form for teachers. Retrieved from http://www.bostonpublicschools.org
Cronbach L. J., 1972, The dependability of behavioral measurements: Theory of generalizability scores and profiles
Danielson Group. (2011). Framework for teaching: Components of professional practice. Retrieved from http://charlottedanielson.com/theframeteach.htm
Gordon R., 2006, Identifying effective teachers using performance on the job
Hill H. C., Herlihy C. (2011). Prioritizing teaching quality in a new system of teacher evaluation. Education Outlook. Retrieved from http://www.aei.org/outlook/101089
Johnson S. M., 1990, Teachers at work: Achieving success in our schools
Measures of Effective Teaching. (n.d.). Retrieved from the Bill and Melinda Gates Foundation website: http://www.gatesfoundation.org/united-states/Pages/measures-of-effective-teaching-fact-sheet.aspx
National Center for Teacher Effectiveness, 2011, Online poll of states engaged in reform of teacher evaluation systems
Peterson K. D., 2000, Teacher evaluation: A comprehensive guide to new directions and practices
Sartain L., 2009, Evaluation of the excellent in teaching pilot: A report to the Joyce Foundation
Scheerens J., 1997, The foundations of educational effectiveness
Shavelson R. J., 1991, Generalizability theory: A primer
Teacher Evaluation Act, La. Rev. Stat. §§ 17:3881-3886, 17:3901-3905, 17:3997, 17:3891-3895 (2010).
Teddlie C., 2000, The international handbook of school effectiveness research
Tennessee Department of Education. (2011). Framework for evaluation and professional growth. Retrieved from http://www.tn.gov/education/frameval/
Weisberg D., 2009, The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness