Examination of the feedback orientation scale in residency selection
Tóm tắt
Giving and receiving feedback is a critical competency for residents. High receptivity to feedback, defined as high feedback orientation, has been shown to improve learning, motivation, and ultimately impact performance. We sought to examine feedback orientation (FO) among general surgery residency applicants and its potential use in screening applicants for residency recruitment. Prospective general surgery residents were asked to complete the Feedback Orientation Scale (FOS) as a part of their application to two academic medical centers. The FOS consisted of a 20-item questionnaire encompassing four dimensions: utility, self-efficacy, social awareness, and accountability. Applicants rated each item using a 5-point scale (1 = strongly disagree, 5 = strongly agree) to create a FOS composite score. Scores were compared to applicant information such as USMLE scores, situational judgment test performance, and self-reported demographic data. In total, 3361 applicants completed the FOS and of these, 2464 self-reported demographic data. Overall, the mean FO score was 4.4 (SD = 0.31) and means across the four dimensions ranged from 4.0 to 4.7, with the lowest in social awareness (4.0, SD = 0.62) and highest in utility (4.7, SD = 0.34). FO scores were associated with USMLE Step 1 scores (0.4, p < 0.05), with highest correlation to utility (0.06, p < 0.001) and accountability (0.06, p < 0.001). In contrast, FO scores were not associated with USMLE Step 2 CK scores (0.008, p = 0.69). However, subdimensions utility and self-efficacy were associated with USMLE Step 2 CK scores when analyzed separately (0.04, p < 0.05 and -0.07, p < 0.001, respectively). Compared to men, women had lower overall FO scores, although the difference was minimal (4.35 vs. 4.39). Asian applicants had slightly higher overall FO scores compared to White applicants (p < 0.001), while other racial and ethnicity comparisons demonstrated no significant differences. To establish convergence validity for use of the FOS, FO scores were found to be positively correlated to each applicant’s SJT scores. High receptivity to feedback is a highly valued attribute among general surgery applicants and can be objectively quantified through use of the FOS. We provide validity evidence for use of this scale in a residency selection setting. Given the recent transition of USMLE Step 1 scoring to pass/fail in January 2022, feedback orientation may be used as an effective adjunct for applicant selection among residency programs.
Tài liệu tham khảo
Ende J. Feedback in clinical medical education. JAMA. 1983;250(6):777–81.
Trehan A, Barnett-Vanes A, Carty MJ, McCulloch P, Maruthappu M. The impact of feedback of intraoperative technical performance in surgery: a systematic review. BMJ Open. 2015;5(6): e006759. https://doi.org/10.1136/bmjopen-2014-006759.
McKendy KM, Watanabe Y, Lee L, et al. Perioperative feedback in surgical training: a systematic review. Am J Surg. 2017;214(1):117–26. https://doi.org/10.1016/j.amjsurg.2016.12.014.
Hajar T, Wanat KA, Fett N. Survey of resident physician and attending physician feedback perceptions: there is still work to be done. Dermatol Online J. 2020. https://doi.org/10.5070/d32512046687.
Nathwani JN, Glarner CE, Law KE, et al. Integrating postoperative feedback into workflow: perceived practices and barriers. J Surg Educ. 2017;74(3):406–14. https://doi.org/10.1016/j.jsurg.2016.11.001.
Chaou C-H, Chang Y-C, Yu S-R, et al. Clinical learning in the context of uncertainty: a multi-center survey of emergency department residents’ and attending physicians’ perceptions of clinical feedback. BMC Med Educ. 2019;19(1):174. https://doi.org/10.1186/s12909-019-1597-8.
Jug R, Jiang XS, Bean SM. Giving and receiving effective feedback: a review article and how-to guide. Arch Pathol Lab Med. 2019;143(2):244–50. https://doi.org/10.5858/arpa.2018-0058-RA.
Gupta A, Villegas CV, Watkins AC, et al. General surgery residents’ perception of feedback: we can do better. J Surg Educ. 2020;77(3):527–33. https://doi.org/10.1016/j.jsurg.2019.12.009.
Jensen AR, Wright AS, Kim S, Horvath KD, Calhoun KE. Educational feedback in the operating room: a gap between resident and faculty perceptions. Am J Surg. 2012;204(2):248–55. https://doi.org/10.1016/j.amjsurg.2011.08.019.
Linderbaum BA, Levy PE. The development and validation of the feedback orientation scale (FOS). J Manag. 2010;36(6):1372–405. https://doi.org/10.1177/0149206310373145.
Dahling JJ, Chau SL, O’Malley A. Correlates and consequences of feedback orientation in organizations. J Manag. 2010;38(2):531–46. https://doi.org/10.1177/0149206310375467.
Gabriel A, Frantz N, Levy P, Hilliard A. The supervisor feedback environment is empowering, but not all the time: feedback orientation as a critical moderator. J Occup Organ Psychol. 2014;87:487–506. https://doi.org/10.1111/joop.12060.
Gardner AK, Cavanaugh KJ, Willis RE, et al. Great expectations? Future competency requirements among candidates entering surgery training. J Surg Educ. 2020;77(2):267–72. https://doi.org/10.1016/j.jsurg.2019.09.001.
National Resident Matching Program DR and RC. Results of the 2021 NRMP program director survey. Washington, D.C.; 2021.
Potts JR. Shifting sands of surgical education. J Am Coll Surg. 2018;227(2):151–62. https://doi.org/10.1016/j.jamcollsurg.2018.02.012.
Potts R, General J. surgery residency: past, present and future. Curr Probl Surg. 2019;56(5):174–97. https://doi.org/10.1067/j.cpsurg.2019.01.005.
The Accreditation Council for Graduate Medical Education. Surgery milestones; 2019.
Cook DA, Hatala R. Validation of educational assessments: a primer for simulation and beyond. Adv Simul. 2016;1(1):31. https://doi.org/10.1186/s41077-016-0033-y.
Gardner AK, Dunkin BJ. Evaluation of validity evidence for personality, emotional intelligence, and situational judgment tests to identify successful residents. JAMA Surg. 2018;153(5):409–16. https://doi.org/10.1001/jamasurg.2017.5013.
Patterson F, Ashworth V, Zibarras L, Coan P, Kerrin M, O’Neill P. Evaluations of situational judgement tests to assess non-academic attributes in selection. Med Educ. 2012;46(9):850–68. https://doi.org/10.1111/j.1365-2923.2012.04336.x.
Gardner AK, Ritter ME, Paige JT, Ahmed RA, Fernandez G, Dunkin BJ. Simulation-based selection of surgical trainees: considerations, challenges, and opportunities. J Am Coll Surg. 2016;223(3):530–6.
Cullen MJ, Zhang C, Marcus-Blank B, et al. Improving our ability to predict resident applicant performance: validity evidence for a situational judgment test. Teach Learn Med. 2020;32(5):508–21. https://doi.org/10.1080/10401334.2020.1760104.
UKCAT Consortium. UCAT university clinical aptitude test. 2019. https://www.ucat.ac.uk.
Vasanthan V, Brown A, Kent W, Holloway D, Maitland A. Virtual adaptation of multimodal cardiac surgery residency interview during covid-19. Can J Cardiol. 2021;37(10):S39–40. https://doi.org/10.1016/j.cjca.2021.07.083.
Lievens F, Patterson F. The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection. J Appl Psychol. 2011;96(5):927–40. https://doi.org/10.1037/a0023496.
Patterson F, Lievens F, Kerrin M, Munro N, Irish B. The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studies. Br J Gen Pract J R Coll Gen Pract. 2013;63(616):e734–41. https://doi.org/10.3399/bjgp13X674413.
Gardner A, Dunkin B. Making progress on identifying those who aren’t making progress: using situational judgment tests to predict those at risk for remediation and attrition [version 1]. MedEdPublish. 2018. https://doi.org/10.15694/mep.2018.0000275.1.
Gardner AK. How can best practices in recruitment and selection improve diversity in surgery? Ann Surg. 2018;267(1):e1–2.
Beckman TJ, Cook DA, Mandrekar JN. What is the validity evidence for assessments of clinical teaching? J Gen Intern Med. 2005;20(12):1159–64. https://doi.org/10.1111/j.1525-1497.2005.0258.x.
Gardner AK, Cavanaugh KJ, Willis RE, Dunkin BJ. Can better selection tools help us achieve our diversity goals in postgraduate medical education? Comparing use of USMLE step 1 scores and situational judgment tests at 7 surgical residencies. Acad Med. 2020;95(5):751–7.
Higgins RSD. Inside baseball-leveling the playing field in the surgical residency selection process. Ann Surg. 2018;267(1):e3.
Edmond MB, Deschenes JL, Eckler M, Wenzel RP. Racial bias in using USMLE step 1 scores to grant internal medicine residency interviews. Acad Med. 2001;76(12):1253–6. https://doi.org/10.1097/00001888-200112000-00021.
Rubright JD, Jodoin M, Barone MA. Examining demographics, prior academic performance, and united states medical licensing examination scores. Acad Med. 2019;94(3):364–70. https://doi.org/10.1097/ACM.0000000000002366.
Brothers TE, Wetherholt S. Importance of the faculty interview during the resident application process. J Surg Educ. 2007;64(6):378–85. https://doi.org/10.1016/j.jsurg.2007.05.003.
Fryer JP, Corcoran N, George B, Wang E, Darosa D. Does resident ranking during recruitment accurately predict subsequent performance as a surgical resident? J Surg Educ. 2012;69(6):724–30. https://doi.org/10.1016/j.jsurg.2012.06.010.
Stohl HE, Hueppchen NA, Bienstock JL. Can medical school performance predict residency performance? Resident selection and predictors of successful performance in obstetrics and gynecology. J Grad Med Educ. 2010;2(3):322–6. https://doi.org/10.4300/JGME-D-09-00101.1.
Sutton E, Richardson JD, Ziegler C, Bond J, Burke-Poole M, McMasters KM. Is USMLE Step 1 score a valid predictor of success in surgical residency? Am J Surg. 2014;208(6):1029–34. https://doi.org/10.1016/j.amjsurg.2014.06.032 (discussion 1034).
McGaghie WC, Cohen ER, Wayne DB. Are United States Medical Licensing Exam Step 1 and 2 scores valid measures for postgraduate medical residency selection decisions? Acad Med. 2011;86(1):48–52. https://doi.org/10.1097/ACM.0b013e3181ffacdb.
Elkbuli A, Kinslow K, Liu H, et al. USMLE scores and clinical rotation role in predicting ABSITE performance among surgery interns. J Surg Res. 2020;247:8–13. https://doi.org/10.1016/j.jss.2019.10.048.
Swanson DB, Sawhill A, Holtzman KZ, et al. Relationship between performance on part I of the American Board of Orthopaedic Surgery certifying examination and scores on USMLE steps 1 and 2. Acad Med. 2009;84(10):21–4.
Pontell ME, Makhoul AT, Ganesh Kumar N, Drolet BC. The change of USMLE step 1 to pass/fail: perspectives of the surgery program director. J Surg Educ. 2021;78(1):91–8. https://doi.org/10.1016/j.jsurg.2020.06.034.
Sergesketter AR, Shammas RL, Langdell HC, et al. Predicting academic performance during plastic surgery residency: can step 2 scores reliably replace step 1? J Surg Educ. 2021. https://doi.org/10.1016/j.jsurg.2021.11.015.