Springer Science and Business Media LLC

This study aimed to investigate whether group self-practice of systematic clinical observation using the airway, breathing, circulation, disability and exposure (ABCDE) approach in a multiplayer, immersive, interactive virtual reality (VR) application provided a non-inferior learning outcome compared to practicing with physical equipment in first-year medical and nursing students. The study was a non-inferior, parallel-group randomized controlled trial. After a 15-min introduction session on the ABCDE approach, all students were randomly allocated to practice ABCDE in groups of three for 20 min either in a fully immersive, interactive, multiplayer virtual reality application (the VR group) or with physical equipment (the TP group). The primary outcome was the number of students who documented all predefined observations in the correct order of the ABCDE approach on a practical test performed immediately after group practice. A total of 84% of all eligible students participated, with 146 students in the VR group and 143 in the TP group. On the primary outcome, 20% in the VR group and 21% in the TP group got everything correct (absolute difference 1% point, one-sided 95% confidence interval 1.0–8.8% points), showing non-inferiority of the virtual reality application. For other outcomes, the results were mostly similar between the groups. Group self-practice of the ABCDE approach in multiplayer, immersive, interactive virtual reality application was non-inferior to practice with physical equipment.

There has been a growing trend in medical education to integrate the use of computers into the undergraduate medical curriculum. While it seems intuitively obvious that personal computers and the Internet can be useful learning tools, it is not clear that the perceived advantages of Computer Assisted Instruction (CAI)are warranted. One problem is that computers are too often used in CAI simply as presentation devices for predefined material without ample consideration paid to the pedagogical principles that have informed more conventional teaching practices. The creation of an environment that is conducive to effective learning has often been overlooked in favour of the development and use of increasingly more sophisticated technologies. The current paper represents an attempt to delineate ways in which we might better develop instructional multimedia programs by employing some of the strategies believed to characterize effective clinical teaching. To do so, this paper will briefly review the work of Irby and others in an attempt to draw attention to ways in which the characteristics identified by these researchers might be implemented for the use of CAI.

Purpose. Post-encounter written exercises (e.g., patient notes) have been included in clinical skills assessments that use standardized patients. The purpose of this study was to estimate the generalizability of the scores from these written exercises when they are rated by various trained health professionals, including physicians. Method. The patient notes from a 10 station clinical skills examination involving 10 first year emergency medicine residents were analytically scored by four rater groups: three physicians, three nurses, three fourth year medical students, three billing clerks. Generalizability analyses were used to partition the various sources of error variance and derive reliability-like coefficients for each group of raters. Results. The generalizability analyses indicated that case-to-case variability was a major source of error variance in the patient note scores. The variance attributable to the rater or to the rater by examinee interaction was negligible. This finding was consistent across the four rater groups. Generalizability coefficients in excess of 0.80 were achieved for each of the four sets of raters. Physicians did, however, produce the most dependable scores. Conclusion. There is little advantage, from a reliability perspective, in using more than one trained physician, or other health professional who is adequately trained to score the patient note. Measurement error is introduced primarily by case sampling variability. This suggests that, if required, increases in the generalizability of the patient note scores can be made through the addition of cases, and not the addition of raters.

Diagnostic errors are a major, largely preventable, patient safety concern. Error interventions cannot feasibly be implemented for every patient that is seen. To identify cases at high risk of error, clinicians should have a good calibration between their perceived and actual accuracy. This experiment studied the impact of feedback on medical interns’ calibration and diagnostic process. In a two-phase experiment, 125 medical interns from Dutch University Medical Centers were randomized to receive no feedback (control), feedback on their accuracy (performance feedback), or feedback with additional information on why a certain diagnosis was correct (information feedback) on 20 chest X-rays they diagnosed in a feedback phase. A test phase immediately followed this phase and had all interns diagnose an additional 10 X-rays without feedback. Outcome measures were confidence–accuracy calibration, diagnostic accuracy, confidence, and time to diagnose. Both feedback types improved overall confidence–accuracy calibration (R2No Feedback = 0.05, R2Performance Feedback = 0.12, R2Information Feedback = 0.19), in line with the individual improvements in diagnostic accuracy and confidence. We also report secondary analyses to examine how case difficulty affected calibration. Time to diagnose did not differ between conditions. Feedback improved interns’ calibration. However, it is unclear whether this improvement reflects better confidence estimates or an improvement in accuracy. Future research should examine more experienced participants and non-visual specialties. Our results suggest that feedback is an effective intervention that could be beneficial as a tool to improve calibration, especially in cases that are not too difficult for learners.

Norm-referenced pass/fail decisions are quite common in achievement testing in health sciences education. The use of relative standards has the advantage of correcting for variations in test-difficulty. However, relative standards also show some serious drawbacks, and the use of an absolute and fixed standard is regularly preferred. The current study investigates the consequences of the use of an absolute instead of a relative standard. The performance of the developed standard setting procedure was investigated by using actual progress test scores obtained at the Maastricht medical school in an episode of eight years. When the absolute instead of the relative standard was used 6% of the decisions changed: 2.6% of the outcomes changed from fail to pass, and 3.4% from pass to fail. The failure rate, which was approximately constant when using the relative standard, varied from 2% to 47% for different tests when an absolute standard was used. It is concluded that an absolute standard is precarious because of the variations in difficulty of tests.

Students cannot learn from feedback unless they pay attention to it. This study investigated relationships between the personal factors of achievement goal orientations, achievement emotions, and attention to feedback in BioWorld, a computer environment for learning clinical reasoning. Novice medical students (N = 28) completed questionnaires to measure their achievement goal orientations and then thought aloud while solving three endocrinology patient cases and reviewing corresponding expert solutions. Questionnaires administered after each case measured participants’ experiences of five feedback emotions: pride, relief, joy, shame, and anger. Attention to individual text segments of the expert solutions was modelled using logistic regression and the method of generalized estimating equations. Participants did not attend to all of the feedback that was available to them. Performance-avoidance goals and shame positively predicted attention to feedback, and performance-approach goals and relief negatively predicted attention to feedback. Aspects of how the feedback was displayed also influenced participants’ attention. Findings are discussed in terms of their implications for educational theory as well as the design and use of computer learning environments in medical education.

Workplace-Based Assessment (WBA) plays a pivotal role in present-day competency-based medical curricula. Validity in WBA mainly depends on how stakeholders (e.g. clinical supervisors and learners) use the assessments—rather than on the intrinsic qualities of instruments and methods. Current research on assessment in clinical contexts seems to imply that variable behaviours during performance assessment of both assessors and learners may well reflect their respective beliefs and perspectives towards WBA. We therefore performed a Q methodological study to explore perspectives underlying stakeholders’ behaviours in WBA in a postgraduate medical training program. Five different perspectives on performance assessment were extracted: Agency, Mutuality, Objectivity, Adaptivity and Accountability. These perspectives reflect both differences and similarities in stakeholder perceptions and preferences regarding the utility of WBA. In comparing and contrasting the various perspectives, we identified two key areas of disagreement, specifically ‘the locus of regulation of learning’ (i.e., self-regulated versus externally regulated learning) and ‘the extent to which assessment should be standardised’ (i.e., tailored versus standardised assessment). Differing perspectives may variously affect stakeholders’ acceptance, use—and, consequently, the effectiveness—of assessment programmes. Continuous interaction between all stakeholders is essential to monitor, adapt and improve assessment practices and to stimulate the development of a shared mental model. Better understanding of underlying stakeholder perspectives could be an important step in bridging the gap between psychometric and socio-constructivist approaches in WBA.

Changes in certification requirements and examinee characteristics are likely to influence the validity of the evidence associated with interpretations made based on test data. We examined whether changes in Educational Commission for Foreign Medical Graduates (ECFMG) certification requirements over time were associated with changes in internal medicine (IM) residency program director ratings and certification examination scores. Comparisons were made between physicians who were ECFMG-certified before and after the Clinical Skills Assessment (CSA) requirement. A multivariate analysis of covariance was conducted to examine the differences in program director ratings based on CSA cohort and whether the examinees emigrated for undergraduate medical education (national vs. international students). A univariate analysis of covariance was conducted to examine differences in scores from the American Board of Internal Medicine (ABIM) Internal Medicine Certification Examination. For both analyses, United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores were used as covariates. Results indicate that, of those certified by ECFMG between 1993 and 1997, 17 % (n = 1,775) left their country of citizenship for undergraduate medical education. In contrast, 38 % (n = 1,874) of those certified between 1999 and 2003 were international students. After adjustment by covariates, the main effect of cohort membership on the program director ratings was statistically significant (Wilks’ λ = 0.99, F 5, 15391 = 19.9, P < 0.001). However, the strength of the relationship between cohort group and the ratings was weak (η = 0.01). The main effect of migration status was statistically significant and weak (Wilks’ λ = 0.98, F 5,15391 = 45.3, P < 0.01; η = 0.02). Differences in ABIM Internal Medicine Certification Examination scores based on whether or not CSA were required was statistically significant, although the magnitude of the association between these variables was very small. The findings suggest that the implementation of an additional evaluation of skills (e.g., history-taking, physical examination) as a prerequisite to postgraduate medical education (residency) provides some additional, relevant data to those who select ECFMG-certified residents.

Background: The ability of physicians to critically reflect on their professional practice has been increasingly valued. Previous research brought to light the multidimensional structure of reflective practice in medicine. It comprises at least five sets of behaviours in response to complex medical problems encountered in professional practice. Factors associated to reflective practice among physicians have, as far as we know, not yet been explored by empirical study. Purpose: To study factors correlated to reflective practice among physicians. Methods: A questionnaire exploring characteristics of professional practice and educational experiences was administered to primary health care physicians. Measurements were related to scores on a reflective practice measuring instrument developed previously. Associations between variables were examined by statistical analysis with tests of correlation and analysis of variance. Results: Reflective practice is negatively correlated to physician’s age and number of years of clinical practice. Working mainly in hospitals and attendance to medical residency programmes in some specialties apparently have a positive effect on reflective practice. Conclusion: Reflective practice tends to decrease with experience. Findings are consistent with the literature on medical expertise that shows a decline of analytical reasoning in proportion to the increase in experience. Some specialty programmes seems to enhance concerns with the scientific basis to professional practice, thereby favouring reflective approaches. Local features of primary health care settings probably explain their negative effect on reflective practice. Strategies to develop reflective practice among physicians should be explored by further research.