Evaluating the impact of malleable factors on percent time lecturing in gateway chemistry, mathematics, and physics courses

Andrews, T. C., & Lemons, P. P. (2015). It’s personal: biology instructors prioritize personal evidence over empirical evidence in teaching decisions. CBE Life Sciences Education, 14(1), ar7. https://doi.org/10.1187/cbe.14-05-0084 Apkarian, N., Henderson, C., Stains, M., Raker, J., Johnson, E., & Dancy, M. (2021). What really impacts the use of active learning in undergraduate STEM education? Results from a national survey of chemistry, mathematics, and physics instructors. PLoS ONE, 16(2), e0247544. https://doi.org/10.1371/journal.pone.0247544 Apkarian, N., & Kirin, D. (2017). Progress through calculus: Census survey technical report. Mathematical Association of America. https://www.maa.org/sites/default/files/PtC%20Technical%20Report_Final.pdf Apkarian, N., & Rasmussen, C. (2017). Mathematics instruction leadership in undergraduate departments. In A. Weinberg, C. Rasmussen, J. Rabin, M. Wawro, & S. Brown, Proceedings of the 20th Annual Conference on Research in Undergraduate Mathematics Education. http://sigmaa.maa.org/rume/RUME20.pdf Apkarian, N., Smith, W. M., Vroom, K., Voigt, M., Gehrtz, J., Team, P. P., & Team, S. P. (2019). X-PIPS-M survey suite. https://www.maa.org/sites/default/files/XPIPSM%20Summary%20Document.pdf Aragón, O. R., Eddy, S. L., & Graham, M. J. (2018). Faculty beliefs about intelligence are related to the adoption of active-learning practices. CBE Life Sciences Education, 17(3), ar47. https://doi.org/10.1187/cbe.17-05-0084 Armbruster, P., Patel, M., Johnson, E., & Weiss, M. (2009). Active learning and student-centered pedagogy improve student attitudes and performance in introductory biology. CBE Life Sciences Education, 8(3), 203–213. https://doi.org/10.1187/cbe.09-03-0025 Austin, A. E., & Sorcinelli, M. D. (2013). The future of faculty development: where are we going? New Directions for Teaching and Learning, 2013(133), 85–97. https://doi.org/10.1002/tl.20048 Bahník, Š, & Vranka, M. A. (2017). Growth mindset is not associated with scholastic aptitude in a large sample of university applicants. Personality and Individual Differences, 117, 139–143. https://doi.org/10.1016/j.paid.2017.05.046 Baker, L. A., Chakraverty, D., Columbus, L., Feig, A. L., Jenks, W. S., Pilarz, M., Stains, M., Waterman, R., & Wesemann, J. L. (2014). Cottrell Scholars Collaborative New Faculty Workshop: professional development for new chemistry faculty and initial assessment of its efficacy. Journal of Chemical Education, 91(11), 1874–1881. https://doi.org/10.1021/ed500547n Ballen, C. J., Wieman, C., Salehi, S., Searle, J. B., & Zamudio, K. R. (2017). Enhancing diversity in undergraduate science: self-efficacy drives performance gains with active learning. CBE Life Sciences Education, 16(4), ar56. https://doi.org/10.1187/cbe.16-12-0344 Bates, D., Mächler, M., Bolker, B. M., & Walker, S. C. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01 Bathgate, M. E., Aragón, O. R., Cavanagh, A. J., Frederick, J., & Graham, M. J. (2019a). Supports: a key factor in faculty implementation of evidence-based teaching. CBE Life Sciences Education, 18(2), ar22. https://doi.org/10.1187/cbe.17-12-0272 Bathgate, M. E., Aragón, O. R., Cavanagh, A. J., Waterhouse, J. K., Frederick, J., & Graham, M. J. (2019b). Perceived supports and evidence-based teaching in college STEM. International Journal of STEM Education, 6(1), 11. https://doi.org/10.1186/s40594-019-0166-3 Bauer, C., Libby, R. D., Scharberg, M., & Reider, D. (2013). Transformative research-based pedagogy workshops for chemistry graduate students and postdocs. Journal of College Science Teaching, 43(2), 36–43. Bazett, T., & Clough, C. L. (2021). Course coordination as an avenue to departmental culture change. Primus, 31(3–5), 467–482. https://doi.org/10.1080/10511970.2020.1793853 Beatty, I. D., Gerace, W. J., Leonard, W. J., & Dufresne, R. J. (2005). Designing effective questions for classroom response system teaching. American Journal of Physics, 74(1), 31–39. https://doi.org/10.1119/1.2121753 Beichner, R. J. (2008). The SCALE-UP project: A student-centered active learning environment for undergraduate programs. National Academy of Sciences. https://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_072628.pdf Beichner, R. J., Saul, J. M., Abbott, D. S., Morse, J. J., Deardorff, D. L., Allain, R. J., Bonham, S. W., Dancy, M. H., & Risley, J. S. (2007). The Student-Centered Activities for Large Enrollment Undergraduate Programs (SCALE-UP) project (Vol. 1). American Association of Physics Teachers. http://www.per-central.org/document/ServeFile.cfm?ID=4517 Benabentos, R., Hazari, Z., Stanford, J. S., Potvin, G., Marsteller, P., Thompson, K. V., Cassone, V. M., Murasko, D., & Kramer, L. (2020). Measuring the implementation of student-centered teaching strategies in lower- and upper-division STEM courses. Journal of Geoscience Education, 69(4), 342–356. https://doi.org/10.1080/10899995.2020.1768005 Bettinger, E., Ludvigsen, S., Rege, M., Solli, I. F., & Yeager, D. (2018). Increasing perseverance in math: evidence from a field experiment in Norway. Journal of Economic Behavior & Organization, 146, 1–15. https://doi.org/10.1016/j.jebo.2017.11.032 Bian, L., Leslie, S.-J., Murphy, M. C., & Cimpian, A. (2018). Messages about brilliance undermine women’s interest in educational and professional opportunities. Journal of Experimental Social Psychology, 76, 404–420. https://doi.org/10.1016/j.jesp.2017.11.006 Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: a longitudinal study and an intervention. Child Development, 78(1), 246–263. https://doi.org/10.1111/j.1467-8624.2007.00995.x Bodzin, A. M., & Beerer, K. M. (2003). Promoting inquiry-based science instruction: the validation of the science teacher inquiry rubric (stir). Journal of Elementary Science Education, 15(2), 39. https://doi.org/10.1007/BF03173842 Borda, E., Schumacher, E., Hanley, D., Geary, E., Warren, S., Ipsen, C., & Stredicke, L. (2020). Initial implementation of active learning strategies in large, lecture STEM courses: lessons learned from a multi-institutional, interdisciplinary STEM faculty development program. International Journal of STEM Education, 7(1), 4. https://doi.org/10.1186/s40594-020-0203-2 Bressoud, D., & Rasmussen, C. (2015). Seven characteristics of successful calculus programs. Notices of the American Mathematical Society, 62(2), 144–146. https://doi.org/10.1090/noti1209 Broda, M., Yun, J., Schneider, B., Yeager, D. S., Walton, G. M., & Diemer, M. (2018). Reducing inequality in academic success for incoming college students: a randomized trial of growth mindset and belonging interventions. Journal of Research on Educational Effectiveness, 11(3), 317–338. https://doi.org/10.1080/19345747.2018.1429037 Brownell, S. E., & Tanner, K. D. (2012). Barriers to faculty pedagogical change: lack of training, time, incentives, and tensions with professional identity? CBE Life Sciences Education, 11(4), 339–346. https://doi.org/10.1187/cbe.12-09-0163 Burke, K. A., Greenbowe, T. J., & Gelder, J. I. (2004). The multi-initiative dissemination project workshops: who attends them and how effective are they? Journal of Chemical Education, 81(6), 897. https://doi.org/10.1021/ed081p897 Burnette, J. L., O’Boyle, E. H., VanEpps, E. M., Pollack, J. M., & Finkel, E. J. (2013). Mind-sets matter: a meta-analytic review of implicit theories and self-regulation. Psychological Bulletin, 139(3), 655–701. https://doi.org/10.1037/a0029531 Canning, E. A., Muenks, K., Green, D. J., & Murphy, M. C. (2019). STEM faculty who believe ability is fixed have larger racial achievement gaps and inspire less student motivation in their classes. Science Advances, 5(2), eaau4734. https://doi.org/10.1126/sciadv.aau4734 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates Inc. https://doi.org/10.4324/9780203771587 Cohen, J. (1992). A power primer. Pyschological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155 Cohen, J., & Goldhaber, D. (2016). Building a more complete understanding of teacher evaluation using classroom observations. Educational Researcher, 45(6), 378–387. https://doi.org/10.3102/0013189X16659442 Common Data Set Initiative. (n.d.). Retrieved 6 July 2021 from https://commondataset.org Coppola, B. P. (2016). Broad & capacious: a new norm for instructional development in a research setting. Change the Magazine of Higher Learning, 48(2), 34–43. https://doi.org/10.1080/00091383.2016.1163206 Cotner, S., Loper, J., Walker, J. D., & Brooks, D. C. (2013). “It’s not you, it’s the room”—are the high-tech, active learning classrooms worth it? Journal of College Science Teaching, 42(6), 82–88. Cox, B. E., McIntosh, K. L., Reason, R. D., & Terenzini, P. T. (2011). A culture of teaching: policy, perception, and practice in higher education. Research in Higher Education, 52(8), 808–829. https://doi.org/10.1007/s11162-011-9223-6 Dancy, M., Lau, A. C., Rundquist, A., & Henderson, C. (2019). Faculty online learning communities: a model for sustained teaching transformation. Physical Review Physics Education Research, 15(2), 020147. https://doi.org/10.1103/PhysRevPhysEducRes.15.020147 Derting, T. L., Ebert-May, D., Henkel, T. P., Maher, J. M., Arnold, B., & Passmore, H. A. (2016). Assessing faculty professional development in STEM higher education: sustainability of outcomes. Science Advances, 2(3), e1501422. https://doi.org/10.1126/sciadv.1501422 Desimone, L. M., & Pak, K. (2017). Instructional coaching as high-quality professional development. Theory into Practice, 56(1), 3–12. https://doi.org/10.1080/00405841.2016.1241947 Drinkwater, M. J., Gannaway, D., Sheppard, K., Davis, M. J., Wegener, M. J., Bowen, W. P., & Corney, J. F. (2014). Managing active learning processes in large first year physics classes: the advantages of an integrated approach. Teaching & Learning Inquiry THe ISSOTL Journal, 2(2), 75–90. https://doi.org/10.2979/teachlearninqu.2.2.75 Durham, M. F., Knight, J. K., Bremers, E. K., DeFreece, J. D., Paine, A. R., & Couch, B. A. (2018). Student, instructor, and observer agreement regarding frequencies of scientific teaching practices using the Measurement Instrument for Scientific Teaching-Observable (MISTO). International Journal of STEM Education, 5, 31. https://doi.org/10.1186/s40594-018-0128-1 Dweck, C. S. (1999). Self-theories: their role in motivation, personality, and development. Psychology Press. Dweck, C. S. (2019). The choice to make a difference. Perspectives on Psychological Science, 14(1), 21–25. https://doi.org/10.1177/1745691618804180 Dweck, C. S., Chiu, C.-Y., & Hong, Y.-Y. (1995). Implicit theories and their role in judgments and reactions: a word from two perspectives. Psychological Inquiry, 6(4), 267–285. https://doi.org/10.1207/s15327965pli0604_1 Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95(2), 256–273. https://doi.org/10.1037/0033-295X.95.2.256 Ebert-May, D., Derting, T. L., Henkel, T. P., Middlemis Maher, J., Momsen, J. L., Arnold, B., & Passmore, H. A. (2015). Breaking the cycle: future faculty begin teaching with learner-centered strategies after professional development. CBE Life Sciences Education, 14(2), ar22. https://doi.org/10.1187/cbe.14-12-0222 Emerson, K. T. U., & Murphy, M. C. (2014). A company I can trust? Organizational lay theories moderate stereotype threat for women. Personality and Social Psychology Bulletin, 41(2), 295–307. https://doi.org/10.1177/0146167214564969 Erdmann, R., Miller, K., & Stains, M. (2020). Exploring STEM postsecondary instructors’ accounts of instructional planning and revisions. International Journal of STEM Education, 7(1), 7. https://doi.org/10.1186/s40594-020-00206-7 Fairweather, J. (2008). Linking evidence and promising practices in science, technology, engineering, and mathematics (STEM) undergraduate education. https://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_072637.pdf Fairweather, J. S., & Rhoads, R. A. (1995). Teaching and the faculty role: enhancing the commitment to instruction in American colleges and universities. Educational Evaluation and Policy Analysis, 17(2), 179–194. https://doi.org/10.3102/01623737017002179 Feldman, A. (2000). Decision making in the practical domain: a model of practical conceptual change. Science Education, 84(5), 606–623. Ferrare, J. J. (2019). A multi-institutional analysis of instructional beliefs and practices in gateway courses to the sciences. CBE Life Sciences Education, 18(2), ar26. https://doi.org/10.1187/cbe.17-12-0257 FLEXspace. (2018). Retrieved 1 June 2020 from https://flexspace.org Foote, K., Knaub, A., Henderson, C., Dancy, M., & Beichner, R. J. (2016). Enabling and challenging factors in institutional reform: the case of SCALE-UP. Physical Review Physics Education Research, 12(1), 010103. https://doi.org/10.1103/PhysRevPhysEducRes.12.010103 Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111 Fukawa-Connelly, T., Johnson, E., & Keller, R. (2016). Can math education research improve the teaching of abstract algebra? Notices of the American Mathematical Society, 63(3), 276–281. https://doi.org/10.1090/noti1339 Gardner, G. E., & Jones, M. G. (2011). Pedagogical preparation of the science graduate teaching assistant: challenges and implications. Science Educator, 20(2), 31–41. Gasiewski, J. A., Eagan, M. K., Garcia, G. A., Hurtado, S., & Chang, M. J. (2012). From gatekeeping to engagement: a multicontextual, mixed method study of student academic engagement in introductory STEM courses. Research in Higher Education, 53(2), 229–261. https://doi.org/10.1007/s11162-011-9247-y Gess-Newsome, J., Southerland, S. A., Johnston, A., & Woodbury, S. (2003). Educational reform, personal practical theories, and dissatisfaction: the anatomy of change in college science teaching. American Educational Research Journal, 40(3), 731–767. https://doi.org/10.3102/00028312040003731 Gibbons, R. E., Villafañe, S. M., Stains, M., Murphy, K. L., & Raker, J. R. (2018). Beliefs about learning and enacted instructional practices: an investigation in postsecondary chemistry education. Journal of Research in Science Teaching, 55(8), 1111–1133. https://doi.org/10.1002/tea.21444 Golnabi, A. H., Murray, E., & Su, H. (2021). How precalculus course coordination can impact students’ academic performance. Journal of Higher Education Theory and Practice. https://doi.org/10.33423/jhetp.v21i5.4278 Gormally, C., Evans, M., & Brickman, P. (2014). Feedback about teaching in higher ed: neglected opportunities to promote change. CBE Life Sciences Education, 13(2), 187–199. https://doi.org/10.1187/cbe.13-12-0235 Haak, D. C., HilleRisLambers, J., Pitre, E., & Freeman, S. (2011). Increased structure and active learning reduce the achievement gap in introductory biology. Science, 332(6034), 1213. https://doi.org/10.1126/science.1204820 Handelsman, J., Ebert-May, D., Beichner, R., Bruns, P., Chang, A., DeHaan, R., Gentile, J., Lauffer, S., Stewart, J., Tilghman, S. M., & Wood, W. B. (2004). Scientific teaching. Science, 304(5670), 521. https://doi.org/10.1126/science.1096022 Harris, R. B., Mack, M. R., Bryant, J., Theobald, E. J., & Freeman, S. (2020). Reducing achievement gaps in undergraduate general chemistry could lift underrepresented students into a “hyperpersistent zone.” Science Advances, 6(24), eaaz5687. https://doi.org/10.1126/sciadv.aaz5687 Hayward, C. N., Weston, T., & Laursen, S. L. (2018). First results from a validation study of TAMI: Toolkit for Assessing Mathematics Instruction. In A. Weinberg, C. Rasmussen, J. Rabin, M. Wawro, & S. Brown, Proceedings of the 21st Annual Conference on Research in Undergraduate Mathematics Education. http://sigmaa.maa.org/rume/RUME21.pdf Henderson, C. (2008). Promoting instructional change in new faculty: an evaluation of the physics and astronomy new faculty workshop. American Journal of Physics, 76(2), 179–187. https://doi.org/10.1119/1.2820393 Henderson, C., Beach, A., & Famiano, M. (2009). Promoting instructional change via co-teaching. American Journal of Physics, 77(3), 274–283. https://doi.org/10.1119/1.3033744 Henderson, C., Beach, A., & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: an analytic review of the literature. Journal of Research in Science Teaching, 48(8), 952–984. https://doi.org/10.1002/tea.20439 Henderson, C., Connolly, M., Dolan, E. L., Finkelstein, N., Franklin, S., Malcom, S., Rasmussen, C., Redd, K., & St. John, K. (2017). Towards the STEM DBER alliance: why we need a discipline-based STEM education research community. International Journal of STEM Education, 4(1), 14. https://doi.org/10.1186/s40594-017-0076-1 Henderson, C., & Dancy, M. H. (2007). Barriers to the use of research-based instructional strategies: the influence of both individual and situational characteristics. Physical Review Physics Education Research, 3, 020102. https://doi.org/10.1103/PhysRevSTPER.3.020102 Henderson, C., & Dancy, M. H. (2009). Impact of physics education research on the teaching of introductory quantitative physics in the united states. Physical Review Special Topics Physics Education Research, 5(2), 020107. https://doi.org/10.1103/PhysRevSTPER.5.020107 Henderson, C., Dancy, M., & Niewiadomska-Bugaj, M. (2012). Use of research-based instructional strategies in introductory physics: where do faculty leave the innovation-decision process? Physical Review Physics Education Research, 8(2), 020104. https://doi.org/10.1103/PhysRevSTPER.8.020104 Herrington, D. G., Bancroft, S. F., Edwards, M. M., & Schairer, C. J. (2016). I want to be the inquiry guy! How research experiences for teachers change beliefs, attitudes, and values about teaching science as inquiry. Journal of Science Teacher Education, 27(2), 183–204. https://doi.org/10.1007/s10972-016-9450-y Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: teacher observation systems and a case for the generalizability study. Educational Researcher, 41(2), 56–64. https://doi.org/10.3102/0013189X12437203 Hora, M. T. (2012). Organizational factors and instructional decision-making: a cognitive perspective. The Review of Higher Education, 35(2), 207–235. https://doi.org/10.1353/rhe.2012.0001 Hora, M. T., & Anderson, C. (2012). Perceived norms for interactive teaching and their relationship to instructional decision-making: a mixed methods study. Higher Education, 64(4), 573–592. https://doi.org/10.1007/s10734-012-9513-8 Houseknecht, J. B., Bachinski, G. J., Miller, M. H., White, S. A., & Andrews, D. M. (2020). Effectiveness of the active learning in organic chemistry faculty development workshops. Chemistry Education Research and Practice, 21(1), 387–398. https://doi.org/10.1039/C9RP00137A Johnson, E., Keller, R., & Fukawa-Connelly, T. (2018). Results from a survey of abstract algebra instructors across the united states: understanding the choice to (not) lecture. International Journal of Research in Undergraduate Mathematics Education, 4(2), 254–285. https://doi.org/10.1007/s40753-017-0058-1 Johnson, E., Keller, R., Peterson, V., & Fukawa-Connelly, T. (2019). Individual and situational factors related to undergraduate mathematics instruction. International Journal of STEM Education, 6(1), 23. https://doi.org/10.1186/s40594-019-0175-2 Knaub, A. V., Foote, K. T., Henderson, C., Dancy, M., & Beichner, R. J. (2016). Get a room: the role of classroom space in sustained implementation of studio style instruction. International Journal of STEM Education, 3(1), 8. https://doi.org/10.1186/s40594-016-0042-3 Koch, A. K. (2017). It’s about the gateway courses: defining and contextualizing the issue. New Directions for Higher Education, 2017(180), 11–17. https://doi.org/10.1002/he.20257 Kogan, M., & Laursen, S. L. (2014). Assessing long-term effects of inquiry-based learning: a case study from college mathematics. Innovative Higher Education, 39(3), 183–199. https://doi.org/10.1007/s10755-013-9269-9 Koziol, S. M., & Burns, P. (1986). Teachers’ accuracy in self-reporting about instructional practices using a focused self-report inventory. The Journal of Educational Research, 79(4), 205–209. https://doi.org/10.1080/00220671.1986.10885678 Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest package: tests in linear mixed effects models. Journal of Statistical Software, 82(13), 1–26. https://doi.org/10.18637/jss.v082.i13 Landis, C. R., Peace, G. E., Scharberg, M. A., Branz, S., Spencer, J. N., Ricci, R. W., Zumdahl, S. A., & Shaw, D. (1998). The new traditions consortium: shifting from a faculty-centered paradigm to a student-centered paradigm. Journal of Chemical Education, 75(6), 741. https://doi.org/10.1021/ed075p741 Landrum, R. E., Viskupic, K., Shadle, S. E., & Bullock, D. (2017). Assessing the STEM landscape: the current instructional climate survey and the evidence-based instructional practices adoption scale. International Journal of STEM Education, 4(1), 25. https://doi.org/10.1186/s40594-017-0092-1 Lane, A. K., McAlpin, J. D., Earl, B., Feola, S., Lewis, J. E., Mertens, K., Shadle, S. E., Skvoretz, J., Ziker, J. P., Couch, B. A., Prevost, L. B., & Stains, M. (2020). Innovative teaching knowledge stays with users. Proceedings of the National Academy of Sciences, 117(37), 22665. https://doi.org/10.1073/pnas.2012372117 Lane, A. K., Skvoretz, J., Ziker, J. P., Couch, B. A., Earl, B., Lewis, J. E., McAlpin, J. D., Prevost, L. B., Shadle, S. E., & Stains, M. (2019). Investigating how faculty social networks and peer influence relate to knowledge and use of evidence-based teaching practices. International Journal of STEM Education, 6(1), 28. https://doi.org/10.1186/s40594-019-0182-3 Leslie, S.-J., Cimpian, A., Meyer, M., & Freeland, E. (2015). Expectations of brilliance underlie gender distributions across academic disciplines. Science, 347(6219), 262–265. https://doi.org/10.1126/science.1261375 Levy, R. (2019). VITAL faculty: a growing workforce in colleges and universities. Mathematical Association of America. Retrieved 10 July 2021 from https://www.mathvalues.org/masterblog/vital-faculty Li, Y., & Bates, T. C. (2017). Does growth mindset improve children’s IQ, educational attainment or response to setbacks? Active-control interventions and data on children’s own mindsets. SocArXiv. https://doi.org/10.31235/osf.io/tsdwy Lorah, J. (2018). Effect size measures for multilevel models: definition, interpretation, and times example. Large-Scale Assessments in Education, 6(1), 8. https://doi.org/10.1186/s40536-018-0061-2 Lorenzo, M., Crouch, C. H., & Mazur, E. (2006). Reducing the gender gap in the physics classroom. American Journal of Physics, 74(2), 118–122. https://doi.org/10.1119/1.2162549 Lotter, C., Harwood, W. S., & Bonner, J. J. (2007). The influence of core teaching conceptions on teachers’ use of inquiry teaching practices. Journal of Research in Science Teaching, 44(9), 1318–1347. https://doi.org/10.1002/tea.20191 Lund, T. J., Pilarz, M., Velasco, J. B., Chakraverty, D., Rosploch, K., Undersander, M., & Stains, M. (2015). The best of both worlds: building on the copus and rtop observation protocols to easily and reliably measure various levels of reformed instructional practice. CBE Life Sciences Education, 14(2), ar18. https://doi.org/10.1187/cbe.14-10-0168 Lund, T. J., & Stains, M. (2015). The importance of context: an exploration of factors influencing the adoption of student-centered teaching among chemistry, biology, and physics faculty. International Journal of STEM Education, 2(1), 13. https://doi.org/10.1186/s40594-015-0026-8 Marsh, H. W., Rowe, K. J., & Martin, A. (2002). Phd students’ evaluations of research supervision: Issues, complexities, and challenges in a nationwide Australian experiment in benchmarking universities. The Journal of Higher Education, 73(3), 313–348. https://doi.org/10.1353/jhe.2002.0028 Martinez, A. E., & Pilgrim, M. E. (2021). Online grading platforms: establishing a connection between course coordination and local data. International Journal of Mathematical Education in Science and Technology. https://doi.org/10.1080/0020739X.2021.1937731 McMahon, K., Yeh, C.S.-H., & Etchells, P. J. (2019). The impact of a modified initial teacher education on challenging trainees’ understanding of neuromyths. Mind, Brain, and Education, 13(4), 288–297. https://doi.org/10.1111/mbe.12219 Michael, J. (2007). Faculty perceptions about barriers to active learning. College Teaching, 55(2), 42–47. Moore, G. A. (2002). Crossing the chasm. HarperCollins. Murray, T. A., Higgins, P., Minderhout, V., & Loertscher, J. (2011). Sustaining the development and implementation of student-centered teaching nationally: the importance of a community of practice. Biochemistry and Molecular Biology Education, 39(6), 405–411. https://doi.org/10.1002/bmb.20537 Myers, R. (1990). Classical and modern regression with applications (2nd ed.). Duxbury. Oleson, A., & Hora, M. T. (2014). Teaching the way they were taught? Revisiting the sources of teaching knowledge and the role of prior experience in shaping faculty teaching practices. Higher Education, 68(1), 29–45. https://doi.org/10.1007/s10734-013-9678-9 Peace, G. E., Lewis, E. L., Burke, K. A., & Greenbowe, T. J. (2002). The multi-initiative dissemination project: active-learning strategies for college chemistry. Journal of Chemical Education, 79(6), 699. https://doi.org/10.1021/ed079p699 Pelletreau, K. N., Knight, J. K., Lemons, P. P., McCourt, J. S., Merrill, J. E., Nehm, R. H., Prevost, L. B., Urban-Lurain, M., & Smith, M. K. (2018). A faculty professional development model that improves student learning, encourages active-learning instructional practices, and works for faculty at multiple institutions. CBE Life Sciences Education, 17(2), es5. https://doi.org/10.1187/cbe.17-12-0260 Popova, M., Kraft, A., Harshman, J., & Stains, M. (2021). Changes in teaching beliefs of early-career chemistry faculty: a longitudinal investigation. Chemistry Education Research and Practice. https://doi.org/10.1039/D0RP00313A Popova, M., Shi, L., Harshman, J., Kraft, A., & Stains, M. (2020). Untangling a complex relationship: teaching beliefs and instructional practices of assistant chemistry faculty at research-intensive institutions. Chemistry Education Research and Practice, 21(2), 513–527. https://doi.org/10.1039/C9RP00217K Porter, S. R., & Umbach, P. D. (2001). Analyzing faculty workload data using multilevel modeling. Research in Higher Education, 42(2), 171–196. https://doi.org/10.1023/A:1026573503271 Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223–231. https://doi.org/10.1002/j.2168-9830.2004.tb00809.x Prosser, M., & Trigwell, K. (1997). Relations between perceptions of the teaching environment and approaches to teaching. British Journal of Educational Psychology, 67(1), 25–35. https://doi.org/10.1111/j.2044-8279.1997.tb01224.x R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/ Rasmussen, C., Ellis, J., Zazkis, D., & Bressoud, D. (2014). Features of successful calculus programs at five doctoral degree granting institutions. Proceedings of the 38th Conference of the International Group for the Psychology of Mathematics Education and the 36th Conference of the North American Chapter of the Psychology of Mathematics Education, Vancouver, Canada. https://www.maa.org/sites/default/files/pdf/cspcc/rasmussen_ellis_zazkis_bressoud.pdf Rasmussen, C., Apkarian, N., Hagman, J. E., Johnson, E., Larsen, S., & Bressoud, D. (2019). Brief report: characteristics of precalculus through calculus 2 programs: insights from a national census survey. Journal for Research in Mathematics Education, 50(1), 98–111. https://doi.org/10.5951/jresematheduc.50.1.0098 Rasmussen, C., & Ellis, J. (2015). Calculus coordination at PhD-granting universities: more than just using the same syllabus, textbook, and final exam. In D. Bressoud, V. Mesa, & C. Rasmussen (Eds.), Insights and recommendations from the maa national study of college calculus (pp. 111–120). MMA Press. Rattan, A., Good, C., & Dweck, C. S. (2012). “It’s ok—not everyone can be good at math”: instructors with an entity theory comfort (and demotivate) students. Journal of Experimental Social Psychology, 48(3), 731–737. https://doi.org/10.1016/j.jesp.2011.12.012 Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: applications and data analysis methods (2nd ed.). Sage. Reinholz, D. L., & Apkarian, N. (2018). Four frames for systemic change in STEM departments. International Journal of STEM Education, 5(1), 3. https://doi.org/10.1186/s40594-018-0103-x Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15(3), 351–357. https://doi.org/10.2307/2087176 Rogers, E. M. (2003). Diffusion of innovations. The Free Press. Ruiz-Primo, M. A., Briggs, D., Iverson, H., Talbot, R., & Shepard, L. A. (2011). Impact of undergraduate science course innovations on learning. Science, 331(6022), 1269. https://doi.org/10.1126/science.1198976 SCALE-UP. (2011). Retrieved 15 July 2020 from https://sites.google.com/view/scale-up/home Selya, A., Rose, J., Dierker, L., Hedeker, D., & Mermelstein, R. (2012). A practical guide to calculating Cohen’s f2, a measure of local effect size, from PROC MIXED. Frontiers in Psychology, 3, 111. https://doi.org/10.3389/fpsyg.2012.00111 Seymour, E., DeWelde, K., & Fry, C. (2011). Determining progress in improving undergraduate STEM education: The reformers’ tale. A White Paper commissioned for the Forum, “Characterizing the Impact and Diffusion of Engineering Education Innovations”. https://www.nae.edu/File.aspx?id=36664 Seymour, E., & Hewitt, N. M. (Eds.). (1997). Talking about leaving: Why undergraduates leave the sciences. Westview Press. https://doi.org/10.1086/495437 Seymour, E., & Hunter, A.-B. (Eds.). (2019). Talking about leaving revisited: persistence, relocation, and loss in undergraduate STEM education. Springer International Publishing. https://doi.org/10.1007/978-3-030-25304-2 Shadle, S. E., Marker, A., & Earl, B. (2017). Faculty drivers and barriers: laying the groundwork for undergraduate STEM education reform in academic departments. International Journal of STEM Education, 4(1), 8. https://doi.org/10.1186/s40594-017-0062-7 Sisk, V. F., Burgoyne, A. P., Sun, J., Butler, J. L., & Macnamara, B. N. (2018). To what extent and under which circumstances are growth mind-sets important to academic achievement? Two Meta-Analyses. Psychological Science, 29(4), 549–571. https://doi.org/10.1177/0956797617739704 Smart, J. C., & Umbach, P. D. (2007). Faculty and academic environments: Using Jolland’s theory to explore differences in how faculty structure undergraduate courses. Journal of College Student Development, 48(2), 183–195. https://doi.org/10.1353/csd.2007.0021 Smeby, J.-C., & Try, S. (2005). Departmental contexts and faculty research activity in Norway. Research in Higher Education, 46(6), 593–619. https://doi.org/10.1007/s11162-004-4136-2 Smith, T., Brumskill, R., Johnson, A., & Zimmer, T. (2018). The impact of teacher language on students’ mindsets and statistics performance. Social Psychology of Education, 21(4), 775–786. https://doi.org/10.1007/s11218-018-9444-z Smith, M. K., Vinson, E. L., Smith, J. A., Lewin, J. D., & Stetzer, M. R. (2014). A campus-wide study of STEM courses: new perspectives on teaching practices and perceptions. CBE Life Sciences Education, 13(4), 624–635. https://doi.org/10.1187/cbe.14-06-0108 Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: an introduction to basic and advanced multilevel modeling (2nd ed.). Sage Publishing. Sonnert, G., Fox, M. F., & Adkins, K. (2007). Undergraduate women in science and engineering: effects of faculty, fields, and institutions over time. Social Science Quarterly, 88(5), 1333–1356. https://doi.org/10.1111/j.1540-6237.2007.00505.x Southerland, S. A., Sowell, S., Blanchard, M., & Granger, E. M. (2011a). Exploring the construct of pedagogical discontentment: A tool to understand science teachers’ openness to reform. Research in Science Education, 41(3), 299–317. https://doi.org/10.1007/s11165-010-9166-5 Southerland, S. A., Sowell, S., & Enderle, P. (2011b). Science teachers’ pedagogical discontentment: Its sources and potential for change. Journal of Science Teacher Education, 22(5), 437–457. https://doi.org/10.1007/s10972-011-9242-3 Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: a meta-analysis. Review of Educational Research, 69(1), 21–51. https://doi.org/10.3102/00346543069001021 Srinivasan, S., Gibbons, R. E., Murphy, K. L., & Raker, J. (2018). Flipped classroom use in chemistry education: results from a survey of postsecondary faculty members. Chemistry Education Research and Practice, 19(4), 1307–1318. https://doi.org/10.1039/C8RP00094H Stains, M., Harshman, J., Barker, M. K., Chasteen, S. V., Cole, R., DeChenne-Peters, S. E., Eagan, M. K., Esson, J. M., Knight, J. K., Laski, F. A., Levis-Fitzgerald, M., Lee, C. J., Lo, S. M., McDonnell, L. M., McKay, T. A., Michelotti, N., Musgrove, A., Palmer, M. S., … (2018). Anatomy of STEM teaching in North American universities. Science, 359(6383), 1468–1470. https://doi.org/10.1126/science.aap8892 Stains, M., Pilarz, M., & Chakraverty, D. (2015). Short and long-term impacts of the Cottrell scholars collaborative new faculty workshop. Journal of Chemical Education, 92(9), 1466–1476. https://doi.org/10.1021/acs.jchemed.5b00324 Stegall, S. L., Grushow, A., Whitnell, R., & Hunnicutt, S. S. (2016). Evaluating the effectiveness of POGIL-PCL workshops. Chemistry Education Research and Practice, 17(2), 407–416. https://doi.org/10.1039/C5RP00225G Sturtevant, H., & Wheeler, L. (2019). The STEM Faculty Instructional Barriers and Identity Survey (FIBIS): development and exploratory results. International Journal of STEM Education, 6, 35. https://doi.org/10.1186/s40594-019-0185-0 Styers, M. L., Van Zandt, P. A., & Hayden, K. L. (2018). Active learning in flipped life science courses promotes development of critical thinking skills. CBE Life Sciences Education, 17(3), ar39. https://doi.org/10.1187/cbe.16-11-0332 Tai, R. H., Sadler, P. M., & Loehr, J. F. (2005). Factors influencing success in introductory college chemistry. Journal of Research in Science Teaching, 42(9), 987–1012. https://doi.org/10.1002/tea.20082 Theobald, E. (2018). Students are rarely independent: when, why, and how to use random effects in discipline-based education research. CBE Life Sciences Education, 17(3), rm2. https://doi.org/10.1187/cbe.17-12-0280 Theobald, E. J., Hill, M. J., Tran, E., Agrawal, S., Arroyo, E. N., Behling, S., Chambwe, N., Cintrón, D. L., Cooper, J. D., Dunster, G., Grummer, J. A., Hennessey, K., Hsiao, J., Iranon, N., Jones, L., Jordt, H., Keller, M., Lacey, M. E., Littlefield, C. E., … (2020). Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences, 117(12), 6476–6483. https://doi.org/10.1073/pnas.1916903117 Tomkin, J. H., Beilstein, S. O., Morphew, J. W., & Herman, G. L. (2019). Evidence that communities of practice are associated with active learning in large STEM lectures. International Journal of STEM Education, 6, 1. https://doi.org/10.1186/s40594-018-0154-z Walczyk, J. J., Ramsey, L. L., & Zha, P. (2007). Obstacles to instructional innovation according to college science and mathematics faculty. Journal of Research in Science Teaching, 44(1), 85–106. https://doi.org/10.1002/tea.20119 Walter, E. M., Beach, A. L., Henderson, C., Williams, C. T., & Ceballos-Madrigal, I. (2021). Understanding conditions for teaching innovation in postsecondary education: development and validation of the Survey of Climate for Instructional Improvement (SCII). International Journal of Technology in Education, 4(2), 166–199. Walter, E. M., Henderson, C. R., Beach, A. L., & Williams, C. T. (2016). Introducing the Postsecondary Instructional Practices Survey (PIPS): a concise, interdisciplinary, and easy-to-score survey. CBE Life Sciences Education, 15(4), ar53. https://doi.org/10.1187/cbe.15-09-0193 Waxman, H. C., & Padrón, Y. N. (2004). The uses of the classroom observation schedule to improve classroom instruction. In H. C. Waxman, R. S. Hilberg, & R. G. Tharp (Eds.), Observational research in U.S. Classrooms: new approaches for understanding cultural and linguistic diversity (pp. 72–96). Cambridge University Press. https://doi.org/10.1017/CBO9780511616419.004 Wheeler, L. B., Maeng, J. L., & Whitworth, B. A. (2017). Characterizing teaching assistants’ knowledge and beliefs following professional development activities within an inquiry-based general chemistry context. Journal of Chemical Education, 94(1), 19–28. https://doi.org/10.1021/acs.jchemed.6b00373 Williams, M., Apkarian, N., Uhing, K., Martinez, A. E., Rasmussen, C., & Smith, W. M. (2021). In the driver’s seat: course coordinators as change agents for active learning in university precalculus to calculus 2. International Journal of Research in Undergraduate Mathematics Education. https://doi.org/10.1007/s40753-021-00153-w Williams, C. T., Walter, E. M., Henderson, C., & Beach, A. L. (2015). Describing undergraduate STEM teaching practices: a comparison of instructor self-report instruments. International Journal of STEM Education, 2(1), 18. https://doi.org/10.1186/s40594-015-0031-y Windschitl, M., & Sahl, K. (2002). Tracing teachers’ use of technology in a laptop computer school: the interplay of teacher beliefs, social dynamics, and institutional culture. American Educational Research Journal, 39(1), 165–205. https://doi.org/10.3102/00028312039001165 Wood, W., & Gentile, J. (2003). Meeting report: the first national academies summer institute for undergraduate education in biology. Cell Biology Education, 2(4), 207–209. https://doi.org/10.1187/cbe.03-10-0042 Woodbury, S., & Gess-Newsome, J. (2002). Overcoming the paradox of change without difference: a model of change in the arena of fundamental school reform. Educational Policy, 16(5), 763–782. https://doi.org/10.1177/089590402237312 Wyse, S. A., Long, T. M., & Ebert-May, D. (2014). Teaching assistant professional development in biology: designed for and driven by multidimensional data. CBE Life Sciences Education, 13(2), 212–223. https://doi.org/10.1187/cbe.13-06-0106 Yeager, D. S., & Dweck, C. S. (2012). Mindsets that promote resilience: when students believe that personal characteristics can be developed. Educational Psychologist, 47(4), 302–314. https://doi.org/10.1080/00461520.2012.722805 Yeager, D. S., Hanselman, P., Walton, G. M., Murray, J. S., Crosnoe, R., Muller, C., Tipton, E., Schneider, B., Hulleman, C. S., Hinojosa, C. P., Paunesku, D., Romero, C., Flint, K., Roberts, A., Trott, J., Iachan, R., Buontempo, J., Yang, S. M., Carvalho, … (2019). A national experiment reveals where a growth mindset improves achievement. Nature, 573(7774), 364–369. https://doi.org/10.1038/s41586-019-1466-y Yeager, D. S., Romero, C., Paunesku, D., Hulleman, C. S., Schneider, B., Hinojosa, C., Lee, H. Y., O’Brien, J., Flint, K., Roberts, A., Trott, J., Greene, D., Walton, G. M., & Dweck, C. S. (2016). Using design thinking to improve psychological interventions: the case of the growth mindset during the transition to high school. Journal of Educational Psychology, 108(3), 374–391. https://doi.org/10.1037/edu0000098 Yeager, D. S., Walton, G. M., Brady, S. T., Akcinar, E. N., Paunesku, D., Keane, L., Kamentz, D., Ritter, G., Duckworth, A. L., Urstein, R., Gomez, E. M., Markus, H. R., Cohen, G. L., & Dweck, C. S. (2016). Teaching a lay theory before college narrows achievement gaps at scale. Proceedings of the National Academy of Sciences, 113(24), E3341. https://doi.org/10.1073/pnas.1524360113