Video-based assessment (VBA) of an open, simulated orthopedic surgical procedure: a pilot study using a single-angle camera to assess surgical skill and decision making
Tóm tắt
Videos have been used in many settings including medical simulation. Limited information currently exists on video-based assessment in surgical training. Effective assessment tools have substantial impact on the future of training. The objectives of this study were as follows: to evaluate the inter-rater reliability of video-based assessment of orthopedic surgery residents performing open cadaveric simulation procedures and to explore the benefits and limitations of video-based assessment. A multi-method technique was used. In the quantitative portion, four residents participated in a Surgical Objective Structured Clinical Examination in 2017 at a quaternary care training center. A single camera bird’s-eye view was used to videotape the procedures. Five orthopedic surgeons evaluated the surgical videos using the Ottawa Surgical Competency Operating Room Evaluation. Interclass correlation coefficient was used to calculate inter-rater reliability. In the qualitative section, semi-structured interviews were used to explore the perceived strengths and limitations of video-based assessment. The scores using video-based assessment demonstrated good inter-rater reliability (ICC = 0.832, p = 0.014) in assessing open orthopedic procedures on cadavers. Qualitatively, the strengths of video-based assessment in this study are its ability to assess global performance and/or specific skills, ability to reassess missed points during live assessment, and potential use for less common procedures. It also allows for detailed constructive feedback, flexible assessment time, anonymous assessment, multiple assessors and serves as a good coaching tool. The main limitations of video-based assessment are poor audio–video quality, and questionable feasibility for assessing readiness for practice. Video-based assessment is a potential adjunct to live assessment in orthopedic open procedures with good inter-rater reliability. Improving audio–video quality will enhance the quality of the assessment and improve the effectiveness of using this tool in surgical training.
Tài liệu tham khảo
van Eaton EG, Tarpley JL, Solorzano CC, Cho CS, Weber SM, Termuhlen PM. Resident education in 2011: three key challenges on the road ahead. Surgery. 2011;149(4):465–73.
Davies J, Khatib M, Bello F. Open surgical simulation—a review. J Surg Educ. 2013;70(5):618–27.
Martin M, Vashisht B, Frezza E, Ferone T, Lopez B, Pahuja M, et al. Competency-based instruction in critical invasive skills improves both resident performance and patient safety. Surgery. 1998;124(2):313–7.
Moore MD, Abelson JS, O’Mahoney P, Bagautdinov I, Yeo H, Watkins AC. Using GoPro to give video-assisted operative feedback for surgery residents: a feasibility and utility assessment. J Surg Educ. 2018;75(2):497–502.
Sharma G, Aycart MA, O’Mara L, Havens J, Nehs M, Shimizu N, et al. A cadaveric procedural anatomy simulation course improves video-based assessment of operative performance. J Surg Res. 2018;223:64–71.
IJgosse WM, Leijte E, Ganni S, Luursema JM, Francis NK, Jakimowicz JJ, et al. Competency assessment tool for laparoscopic suturing: development and reliability evaluation. Surg Endosc [Internet]. 2020;34(7):2947–53. https://doi.org/10.1007/s00464-019-07077-2.
Yeung C, Carrillo B, Pope V, Hosseinpour S, Gerstle JT, Azzie G. Video assessment of laparoscopic skills by novices and experts: implications for surgical education. Surg Endosc [Internet]. 2017;31(10):3883–9. https://doi.org/10.1007/s00464-017-5417-0.
Bilgic E, Endo S, Lebedeva E, Takao M, McKendy KM, Watanabe Y, et al. A scoping review of assessment tools for laparoscopic suturing. Surg Endosc [Internet]. 2018;32(7):3009–23. https://doi.org/10.1007/s00464-018-6199-8.
Koehler RJ, Nicandri GT. Using the arthroscopic surgery skill evaluation tool as a pass-fail examination. J Bone Joint Surg. 2013;95:23.
Alvand A, Khan T, Al-Ali S, Jackson WF, Price AJ, Rees JL. Simple visual parameters for objective assessment of arthroscopic skill. J Bone Joint Surg. 2012;94(13):e97.
Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med [Internet]. 2002;77(5):361–7.
Sonnadara RR, Mui C, McQueen S, Mironova P, Nousiainen M, Safir O, et al. Reflections on competency-based education and training for surgical residents. J Surg Educ. 2014;71(1):151–8.
Tavakol M, Mohagheghi MA, Dennick R. Assessing the skills of surgical residents using simulation. J Surg Educ. 2008;65(2):77–83.
Gillis ME, Scott SA, Richardson CG, Oxner WM, Gauthier L, Wilson DA, et al. Developing and assessing the feasibility of implementing a surgical objective structured clinical skills examination (S-OSCE). J Surg Educ. 2020;77(4):939–46.
Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ. The Ottawa surgical competency operating room evaluation (O-SCORE): a tool to assess surgical competence. Acad Med. 2012;87(10):1401–7.
Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63.
Laeeq K, Infusino S, Lin SY, Reh DD, Kim J, Lane AP, et al. Video-based assessment of operative competency in endoscopic sinus surgery. Am J Rhinol Allergy. 2010;24:234–7.
Driscoll PJ, Paisley AM, Paterson-Brown S. Video assessment of basic surgical trainees’ operative skills. Am J Surg. 2008;196(2):265–72.
Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, et al. Surgical skill and complication rates after bariatric surgery. N Engl J Med [Internet]. 2013;369(15):1434–42. https://doi.org/10.1056/NEJMsa1300625.
Alvand A, Logishetty K, Middleton R, Khan T, Jackson WFM, Price AJ, et al. Validating a global rating scale to monitor individual resident learning curves during arthroscopic knee meniscal repair. Arthrosc J Arthrosc Relat Surg. 2013;29(5):906–12.
van Hove PD, Tuijthof GJM, Verdaasdonk EGG, Stassen LPS, Dankelman J. Objective assessment of technical surgical skills. Br J Surg [Internet]. 2010;97(7):972–87.
Soucisse ML, Boulva K, Sideris L, Drolet P, Morin M, Dubé P. Video coaching as an efficient teaching method for surgical residents—a randomized controlled trial. J Surg Educ. 2017;74(2):365–71.
Naik ND, Abbott EF, Gas BL, Murphy BL, Farley DR, Cook DA. Personalized video feedback improves suturing skills of incoming general surgery trainees. Surgery. 2018;163(4):921–6.
Aljamal Y, Saleem H, Prabhakar N, Abhishek C, Farley DR. Group video feedback is an effective and efficient tool for enhancing skills of surgical interns. J Surg Res. 2020;251:248–53.
Bowles PFD, Harries M, Young P, Das P, Saunders N, Fleming JC. A validation study on the use of intra-operative video recording as an objective assessment tool for core ENT surgery. Clin Otolaryngol [Internet]. 2014;39(2):102–7. https://doi.org/10.1111/coa.12240.
Dath D, Regehr G, Birch D, Schlachta C, Poulin E, Mamazza J, et al. Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills. Surg Endosc Int Techn [Internet]. 2004;18(12):1800–4. https://doi.org/10.1007/s00464-003-8157-2.
Nepple JJ, Larson CM, Smith MV, Kim YJ, Zaltz I, Sierra RJ, et al. The reliability of arthroscopic classification of acetabular rim labrochondral disease. Am J Sports Med [Internet]. 2012;40(10):2224–9. https://doi.org/10.1177/0363546512457157.
Devitt BM, Smith B, Stapf R, Jo S, O’Donnell JM. The reliability of commonly used arthroscopic classifications of ligamentum teres pathology. J Hip Preserv Surg [Internet]. 2017;4(2):187–93.