Clinician educators’ conceptions of assessment in medical education
Tóm tắt
In pursuing assessment excellence, clinician-educators who design and implement assessment are pivotal. The influence of their assessment practice in university-run licensure exams on student learning has direct implications for future patient care. While teaching practice has been shown to parallel conceptions of teaching, we know too little about conceptions of assessment in medical education to know if this is the case for assessment practice and conceptions of assessment. To explore clinician-educators’ conceptions of assessment, a phenomenographic study was undertaken. Phenomenography explores conceptions, the qualitatively different ways of understanding a phenomenon. Data analysis identifies a range of hierarchically inclusive categories of understanding, from simple to more complex, and the dimensions that distinguish each category or conception. Thirty-one clerkship convenors in three diverse Southern settings were interviewed in three cycles of iterative data collection and analysis. Four conceptions of assessment were identified: passive operator, awakening enquirer, active owner and scholarly assessor. Six dimensions were elucidated to describe and distinguish each conception: purpose of assessment; temporal perspective; role and responsibility; accountability; reflexivity and emotional valence. Additionally, three characteristics that appeared to track the progressive nature of the conceptions were identified: professional identity, assessment literacy and self-efficacy. These conceptions encompass and extend previously described conceptions across different educational levels, disciplines and contexts, suggesting applicability to other settings. There is some evidence of a relationship between conceptions and practice, suggesting, together with the hierarchical nature of these conceptions, that targeting conceptions during faculty development may be an effective approach to enhance assessment practice.
Từ khóa
Tài liệu tham khảo
Adendorff, H. (2011). Strangers in a strange land–on becoming scholars of teaching. London Review of Education, 9(3), 305–315. https://doi.org/10.1080/14748460.2011.616323
Åkerlind, G. (2005). Variation and commonality in phenomenographic research methods. Higher Education Research & Development, 24(4), 321–334. https://doi.org/10.1080/07294360500284672
Åkerlind, G. (2018). What future for phenomenographic research? On continuity and development in the phenomenography and variation theory research tradition. Scandinavian Journal of Educational Research, 62(6), 949–958. https://doi.org/10.1080/00313831.2017.1324899
Barnard, A., McCosker, H., & Gerber, R. (1999). Phenomenography: A qualitative research approach for exploring understanding in health care. Qualitative Health Research, 9(2), 212–226.
Bearman, M., Dawson, P., Boud, D., Bennett, S., Hall, M., & Molloy, E. (2016). Support for assessment practice: Developing the assessment design decisions framework. Teaching in Higher Education, 21(5), 545–556. https://doi.org/10.1080/13562517.2016.1160217
Berendonk, C., Stalmeijer, R., & Schuwirth, L. (2013). Expertise in performance assessment: Assessors’ perspectives. Advances in Health Sciences Education, Theory and Practice, 18(4), 559–571. https://doi.org/10.1007/s10459-012-9392-x
Black, P., & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. https://doi.org/10.1080/0969595980050102
Boud, D. (1995) Assessmnf and learning: contradictory or complementary?. In: P. Knight (Ed.), Assessment for learning in higher education. London: Kogan Page, pp. 35–48.
Boud, D., & Dawson, P. (2021). What feedback literate teachers do: An empirically-derived competency framework. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2021.1910928
Box, C., Skoog, G., & Dabbs, J. (2015). A case study of teacher personal practice assessment theories and complexities of implementing formative assessment. American Educational Research Journal, 52(5), 956–983. https://doi.org/10.3102/0002831215587754
Braun, V., & Clarke, V. (2021). To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qualitative Research in Sport, Exercise and Health, 13(2), 201–216. https://doi.org/10.1080/2159676x.2019.1704846
Brown, G. (2004). Teachers’ conceptions of assessment - implications for policy and professional development. Assessment in Education - Principles, Policy and Practice, 11(3), 301–318. https://doi.org/10.1080/0969594042000304609
Cantillon, P., D’Eath, M., de Grave, W., & Dornan, T. (2016). How do clinicians become teachers? A communities of practice perspective. Advances in Health Sciences Education, 21(5), 991–1008. https://doi.org/10.1007/s10459-016-9674-9
Cilliers, F., Schuwirth, L., Adendorff, H., Herman, N., & van der Vleuten, C. (2010). The mechanism of impact of summative assessment on medical students’ learning. Advances in Health Science Education, Theory, and Practice, 15(5), 695–715. https://doi.org/10.1007/s10459-010-9232-9
Cilliers, F., Schuwirth, L., Herman, N., Adendorff, H., & van der Vleuten, C. (2012a). A model of the pre-assessment learning effects of summative assessment in medical education. Advances in Health Science Education, Theory, and Practice, 17(1), 39–53. https://doi.org/10.1007/s10459-011-9292-5
Cilliers, F., Schuwirth, L., & van der Vleuten, C. (2012b). Modelling the pre-assessment learning effects of assessment: Evidence in the validity chain. Medical Education, 46(11), 1087–1098. https://doi.org/10.1111/j.1365-2923.2012.04334.x
Collier-Reed, B., & Ingerman, Å. (2013). Phenomenography: From critical aspects to knowledge claim. In Theory and method in higher education research (pp. 243–260).
Connell, R. (2014). Using southern theory: Decolonizing social thought in theory, research and application. Planning Theory, 13(2), 210–223. https://doi.org/10.1177/1473095213499216
Dahlgren, L. (1975). Qualitative differences in learning as a function of content-oriented guidance. Acta Universitatis Gothoburgensis.
de Jonge, L., Timmerman, A., Govaerts, M., Muris, J., Muijtjens, A., Kramer, A., & van der Vleuten, C. (2017). Stakeholder perspectives on workplace-based performance assessment: Towards a better understanding of assessor behaviour. Advances in Health Science Education, Theory, and Practice. https://doi.org/10.1007/s10459-017-9760-7
DeLuca, C., Coombs, A., & LaPointe-McEwan, D. (2019). Assessment mindset: Exploring the relationship between teacher mindset and approaches to classroom assessment. Studies in Educational Evaluation, 61, 159–169. https://doi.org/10.1016/j.stueduc.2019.03.012
Doja, A., Horsley, T., & Sampson, M. (2014). Productivity in medical education research: An examination of countries of origin. Medical Education, 14, 243.
Dortins, E. (2002) Reflections on phenomenographic process: Interview, transcription and analysis. Higher Education Research and Development Society of Australasia In: Higher Education Research and Development Society of Australasia; Quality conversations; 207–213; 2002. ISBN: 0908557515, 090855754X. ISSN: 0155-6233. Conference paper/Print.
Gibbs, G., & Simpson, C. (2004). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education, 1(1), 3–31.
Gosselin, K., Norris, J., & Ho, M. (2016). Beyond homogenization discourse: Reconsidering the cultural consequences of globalized medical education. Medical Teacher, 38(7), 691–699. https://doi.org/10.3109/0142159X.2015.1105941
Govaerts, M., van de Wiel, M., Schuwirth, L., van der Vleuten, C., & Muijtjens, A. (2013). Workplace-based assessment: Raters’ performance theories and constructs. Advances in Health Sciences Education, Theory and Practice, 18(3), 375–396. https://doi.org/10.1007/s10459-012-9376-x
Halinen, K., Ruohoniemi, M., Katajavuori, N., & Virtanen, V. (2013). Life science teachers’ discourse on assessment: A valuable insight into the variable conceptions of assessment in higher education. Journal of Biological Education, 48(1), 16–22. https://doi.org/10.1080/00219266.2013.799082
Ho, A. (2000). A conceptual change approach to staff development A model for programme design. International Journal for Academic Development, 5(1), 30–41. https://doi.org/10.1080/136014400410088
Ho, A., Watkins, D., & Kelly, M. (2001). The conceptual change approach to improving teaching and learning: an evaluation of a Hong Kong staff development programme. Higher Education, 42(2), 143–169.
Joughin, G. (2010). The hidden curriculum revisited - a critical review of research into the influence of summative assessment on learning. Assessment and Evaluation in Higher Education, 35(5), 335–345. https://doi.org/10.1080/02602930903221493
Kember, D. (1997). A reconceptualisation of the research into university academics’ conceptions of teaching. Learning and Instruction, 7(3), 255–275.
Maggio, L. A., Costello, J. A., Ninkov, A., Frank, J. R., & Artino, A. R. (2022). The voices of medical education science: describing the published landscape. https://doi.org/10.1101/2022.02.10.479930
Marton, F. (1981). Phenomenography – describing conceptions of the world around us. Instructional Science, 10, 177–200.
Marton, F. (1986). Phenomenography—a research approach to investigating different understandings of reality. Journal of Thought, 21(3), 28–49.
Marton, F., & Pong, W. (2005). On the unit of description in phenomenography. Higher Education Research & Development, 24(4), 335–348. https://doi.org/10.1080/07294360500284706
Naidu, T. (2021a). Says who? Northern ventriloquism, or epistemic disobedience in global health scholarship. The Lancet Global Health, 9(9), e1332–e1335. https://doi.org/10.1016/s2214-109x(21)00198-4
Naidu, T. (2021b). Southern exposure: Levelling the Northern tilt in global medical and medical humanities education. Advanced Health Science Education, 26, 739–752. https://doi.org/10.1007/s10459-020-09976-9
Nelson, J. (2017). Using conceptual depth criteria: Addressing the challenge of reaching saturation in qualitative research. Qualitative Research, 17(5), 554–570.
Norton, L., Floyd, S., & Norton, B. (2019). Lecturers’ views of assessment design, marking and feedback in higher education: A case for professionalisation? Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2019.1592110
Offerdahl, E., & Tomanek, D. (2011). Changes in instructors’ assessment thinking related to experimentation with new strategies. Assessment & Evaluation in Higher Education, 36(7), 781–795. https://doi.org/10.1080/02602938.2010.488794
Pang, M., & Ki, W. (2016). Revisiting the Idea of “Critical Aspects.” Scandinavian Journal of Educational Research, 60(3), 323–336. https://doi.org/10.1080/00313831.2015.1119724
Paton, M., Naidu, T., Wyatt, T. R., Oni, O., Lorello, G. R., Najeeb, U., & Kuper, A. (2020). Dismantling the master’s house: New ways of knowing for equity and social justice in health professions education. Advances in Health Sciences Education: Theory and Practice, 25(5), 1107–1126. https://doi.org/10.1007/s10459-020-10006-x
Postareff, L., Virtanen, V., Katajavuori, N., & Lindblom-Ylänne, S. (2012). Academics’ conceptions of assessment and their assessment practices. Studies in Educational Evaluation, 38(3–4), 84–92. https://doi.org/10.1016/j.stueduc.2012.06.003
Rotgans, J. (2011). The themes, institutions, and people of medical education research 1988–2010: Content analysis of abstracts from six journals. Advanced Health Science Education Theory and Practice, 17(4), 515–527. https://doi.org/10.1007/s10459-011-9328-x
Rovio-Johansson, A., & Ingerman, A. (2016). Continuity and development in the phenomenography and variation theory tradition. Scandinavian Journal of Educational Research, 60(3), 257–271. https://doi.org/10.1080/00313831.2016.1148074
Samuelowicz, K., & Bain, J. D. (2002). Identifying academics’ orientations to assessment practice. Higher Education, 43(2), 173–201.
Schuwirth, L. W., & van der Vleuten, C. P. (2019). How ‘testing’ has become ‘programmatic assessment for learning.’ Health Professions Education, 5(3), 177–184.
Sims, D., & Cilliers, F. (in press). Qualitatively speaking: Deciding how much data and analysis is enough. African Journal of Health Professions Education.
Sjostrom, B., & Dahlgren, L. O. (2002). Applying phenomenography in nursing research. Journal of Advanced Nursing, 40(3), 339–345. https://doi.org/10.1046/j.1365-2648.2002.02375.x.
Stenfors-Hayes, T., Hult, H., & Dahlgren, M. (2013). A phenomenographic approach to research in medical education. Medical Education, 47(3), 261–270. https://doi.org/10.1111/medu.12101
Svensson, L. (1977). On qualitative differences in learning III: Study skill and learning. British Journal of Educational Psychology. https://doi.org/10.1111/j.2044-8279.1977.tb02352.x
Svensson, L. (1997). theoretical foundations of phenomenography. Higher Education Research & Development, 16(2), 159–171. https://doi.org/10.1080/0729436970160204
Swan Sein, A., Rashid, H., Meka, J., Amiel, J., & Pluta, W. (2020). Twelve tips for embedding assessment for and as learning practices in a programmatic assessment system. Medical Teacher. https://doi.org/10.1080/0142159X.2020.1789081
Tight, M. (2016). Phenomenography: The development and application of an innovative research design in higher education research. International Journal of Social Research Methodology, 19(3), 319–338. https://doi.org/10.1080/13645579.2015.1010284
Trigwell, K., Prosser, M., & Waterhouse, F. (1999). Relations between teachers’ approaches to teaching and students’ approaches to learning. Higher Education, 37(1), 57–70.
Tutarel, O. (2002). Geographical distribution of publications in the field of medical education. Medical Education, 2, 3. https://doi.org/10.1186/1472-6920-2-3
Watkins, D., Dahlin, B., & Ekholm, M. (2005). Awareness of the backwash effect of assessment- a phenomenographic study of the views of Hong Kong and Swedish lecturers. Instructional Science, 33(4), 283–309.
WHO. (2020). World Health Statistics 2020: Monitoring health for the Sustainable Development Goals.