The impact of a dedicated training program for oral examiners at a medical school in Germany: a survey among participants from operative and non-operative disciplines
Tóm tắt
Oral examinations have been a crucial format in ancient and modern assessment to evaluate and guarantee quality of medical education and thereby to secure patient safety. To achieve a high level of quality in the oral part of the final examination of medical students, a training program for oral examiners at the Medical Faculty of Ulm (Germany) has been established since 2007. However, little is known about the attitude of the examiners in regard to the impact of this training program and of oral examinations as instruments to ensure patient safety. All 367 academic clinicians from operative and non-operative disciplines, attending the one-day examiner training program at the University of Ulm between 2007 and 2012 have been asked to answer an online survey (EvaSys 5.0). Focus of the survey was to find out in which respect the examiners profited from the trainings, if the training effects were discipline-dependent, and to which degree the oral examinations could contribute to patient safety. Statistical analysis was performed using the t-test for independent samples. Results were considered statistically significant when p < 0.05. A total of 63 participants answered the survey, but in 4 cases the questionnaire was not fully completed (with single items missing). More than half of the study participants (n = 34/59; 58%) have experienced (at least sometimes or rarely) candidates that they deemed incompetent and perhaps even dangerous to the patients’ health who nevertheless passed the oral exam successfully. The majority of participants were convinced that oral examinations using concrete clinical cases could significantly contribute to patient safety, if grading is based on clear criteria and if examinations as well as grading are performed more critically. The impact of the training program was rated significantly stronger by surgeons than by non-surgeons in several categories. These categories included “strengths and weaknesses of oral examinations”, “reliability”, “validity”, “competence in grading”, “critical grading”, and “departmental improvements” concerning oral examinations. In respect to patient safety, it seems crucial to prevent incompetent candidates from passing the oral examination. The present study indicates the importance to continue and to develop our examiner trainings, with main emphasis on concrete clinical problems and a criteria-based critical grading system for oral examinations. Since the impact of the training was particularly high for colleagues from the operative disciplines, the training program should be offered especially in surgical departments.
Tài liệu tham khảo
Ärztliche Approbationsordnung (ÄAppO) in der Fassung vom 27. 6. 2002. Bundesgesetzblatt. 2002,http://www.gesetze-im-internet.de/_appro_2002/BJNR240500002.html,
Schulze J, Drolshagen S: Format and implementation of written assessments. GMS Z Med Ausbild. 2006, 23 (3): Doc44-
Van der Vleuten CP: The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ. 1996, 1 (1): 41-67. 10.1007/BF00596229.
Wakeford R, Southgate L, Wass V: Improving oral examinations: selecting, training and monitoring examiners for the MRCGP. BMJ. 1995, 311: 931-935. 10.1136/bmj.311.7010.931.
Möltner A, Schellberg D, Jünger J: Basic quantitative analyses of medical examinations. GMS Z Med Ausbild. 2006, 23 (3): Doc53-
Levine HG, McGuire CH: The validity and reliability of oral examinations in assessing cognitive skills in medicine. J Educ Meas. 1970, 7 (2): 63-74. 10.1111/j.1745-3984.1970.tb00697.x.
Oechsner W, Geiler S, Huber-Lang M: Effects and Sustainability of Trainings for the Oral and Practical Part of the German Final Exam in Medicine. GMS Z Med Ausbild. 2013, in press
Van der Vleuten CP: The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ. 1996, 1 (1): 41-47. 10.1007/BF00596229.
Wass V, Wakeford R, Neighbour R, Vleuten CV: Achieving acceptable reliability in oral examinations: an analysis of the Royal College of General Practitioners membership examination’s oral component. Med Educ. 2003, 37 (2): 126-131. 10.1046/j.1365-2923.2003.01417.x.
Seyfarth M, Reincke M, Seyfarth J, Ring J, Fischer MR: Grades on the Second Medical Licensing Examination in Germany Before and After the Licensing Reform of 2002. A study in Two Medical Schools in Bavaria. Dtsch Arztebl Int. 2010, 107 (28–29): 500-504.
Burchard KW, Rowland PA, Coe NPW, Garb J: A surgery oral examination: Interrater Agreement and the influence of rater characteristics. Acad Med. 1995, 70: 1044-1046. 10.1097/00001888-199511000-00026.
Huber-Lang M, Neugebauer E: The dual role of academic surgeons as clinicians and researchers - an attempt to square the circle?. Patient Saf Surg. 2011, 5: 16-10.1186/1754-9493-5-16.
Sheehan K: E-mail Survey Response Rates: A Review. J Comp-Med Comm JCMC. 2001,http://jcmc.indiana.edu/vol6/issue2/sheehan.html,
Groves RM, Cialdini RB, Courier MP: Understanding the decision to participate in a survey. Public Opin Q. 1992, 56: 475-495. 10.1086/269338.
Nagler M, Feller S, Beyeler C: Retrospektive Anpassung der Selbsteinschätzung ärztlicher Kompetenzen – Beachtenswert bei der Evaluation praktischer Weiterbildungskurse. GMS Z Med Ausbild. 2012, 29 (3): Doc45-
Jünger J, Schellberg D, Nikendei C: Subjektive Kompetenzeinschätzung von Studierenden und ihre Leistung im OSCE. GMS Z Med Ausbild. 2006, 23 (3): Doc51-
Gordon MJ: A review of the validity and accuracy of self-assessments in health professions training. Acad Med. 1991, 66 (12): 762-769. 10.1097/00001888-199112000-00012.