Observation of behavioural skills by medical simulation facilitators: a cross-sectional analysis of self-reported importance, difficulties, observation strategies and expertise development

Advances in Simulation - 2023
Lars Mommers1,2, Daniëlle Verstegen3, Diana Dolmans3, Walther N. K. A. van Mook4,3,5
1Department of Anaesthesiology and Pain Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands
2Department of Simulation in Healthcare, Maastricht University Medical Centre, Maastricht, The Netherlands
3School of Health Professions Education, Maastricht University, Maastricht, The Netherlands
4Department of Intensive Care Medicine, Maastricht University Medical Centre+, Maastricht, the Netherlands
5Department of Postgraduate Medical Training, Maastricht University Medical Centre, Maastricht, The Netherlands

Tóm tắt

The association between team performance and patient care was an immense boost for team-based education in health care. Behavioural skills are an important focus in these sessions, often provided via a mannikin-based immersive simulation experience in a (near) authentic setting. Observation of these skills by the facilitator(s) is paramount for facilitated feedback with the team. Despite the acknowledgement that trained facilitators are important for optimal learning, insight into this observation process by facilitators is limited. What are the self-reported current practices and difficulties regarding the observation of behavioural skills amongst facilitators during team training and how have they been trained to observe behavioural skills? This cross-sectional study used a pilot-tested, content-validated, multi-linguistic online survey within Europe, distributed through a non-discriminative snowball sampling method. Inclusion was limited to facilitators observing behavioural skills within a medical team setting. A total of 175 persons filled in the questionnaire. All aspects of behavioural skill were perceived as very important to observe. The self-perceived difficulty of the behavioural skill aspects ranged from slightly to moderately difficult. Qualitative analysis revealed three major themes elaborating on this perceived difficulty: (1) not everything can be observed, (2) not everything is observed and (3) interpretation of observed behavioural skills is difficult. Additionally, the number of team members health care facilitators have to observe, outnumbers their self-reported maximum. Strategies and tools used to facilitate their observation were a blank notepad, co-observers and predefined learning goals. The majority of facilitators acquired observational skills through self-study and personal experience and/or observing peers. Co-observation with either peers or experts was regarded as most learn some for their expertise development. Overall, participants perceived themselves as moderately competent in the observation of behavioural skills during team training. Observation of behavioural skills by facilitators in health care remains a complex and challenging task. Facilitators’ limitations with respect to attention, focus and (in)ability to perform concomitant tasks, need to be acknowledged. Although strategies and tools can help to facilitate the observation process, they all have their limitations and are used in different ways.

Từ khóa


Tài liệu tham khảo

Mazzocco K, Petitti DB, Fong KT, Bonacum D, Brookey J, Graham S, et al. Surgical team behaviors and patient outcomes. Am J Surg. 2009;197(5):678–85.

Siu J, Maran N, Paterson-Brown S. Observation of behavioural markers of non-technical skills in the operating room and their relationship to intra-operative incidents. Surgeon. 2016;14(3):119–28.

Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery. 2003;133(6):614–21.

Institute of Medicine Committee on Quality of Health Care in A. To err is human: building a safer health system. In: Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. Washington (DC): National Academies Press (US) Copyright 2000 by the National Academy of Sciences. All rights reserved; 2000.

Dedy NJ, Bonrath EM, Zevin B, Grantcharov TP. Teaching nontechnical skills in surgical residency: a systematic review of current approaches and outcomes. Surgery. 2013;154(5):1000–8.

Flin R, O’Connor P, Crichton M. Safety at the sharp end. A guide to non-technical skills. Surrey: Ashgate; 2008. p. 336.

Flin R, Maran N. Basic concepts for crew resource management and non-technical skills. Best Pract Res Clin Anaesthesiol. 2015;29(1):27–39.

Jepsen RMHG, Østergaard D, Dieckmann P. Development of instruments for assessment of individuals’ and teams’ non-technical skills in healthcare: a critical review. Cogn Technol Work. 2015;17(1):63–77.

Murphy P, Nestel D, Gormley GJ. Words matter: towards a new lexicon for 'nontechnical skills’ training. Adv Simul (Lond). 2019;4:8.

Sevdalis N, Hull L, Birnbach DJ. Improving patient safety in the operating theatre and perioperative care: obstacles, interventions, and priorities for accelerating progress. Br J Anaesth. 2012;109(Suppl 1):i3–16.

Schmitt M, Blue A, Aschenbrener CA, Viggiano TR. Core competencies for interprofessional collaborative practice: reforming health care by transforming health professionals’ education. Acad Med. 2011;86(11):1351.

Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand. 2009;53(2):143–51.

Julia Neily J, Mills PD, Young-Xu Y, Carney BT, West P, Berger DH, et al. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304(15):1693–700.

Catchpole KR, Dale TJ, Hirst DG, Smith JP, Giddings TA. A multicenter trial of aviation-style training for surgical teams. J Patient Saf. 2010;6(3):180–6.

Marshall DA, Manus DA. A team training program using human factors to enhance patient safety. AORN J. 2007;86(6):994–1011.

Awad SS, Fagan SP, Bellows C, Albo D, Green-Rashad B, De la Garza M, et al. Bridging the communication gap in the operating room with medical team training. Am J Surg. 2005;190(5):770–4.

Carpenter JE, Bagian JP, Snider RG, Jeray KJ. Medical team training improves team performance: AOA critical issues. J Bone Joint Surg Am. 2017;99(18):1604–10.

Kolbe M, Grande B, Spahn DR. Briefing and debriefing during simulation-based training and beyond: content, structure, attitude and setting. Best Pract Res Clin Anaesthesiol. 2015;29(1):87–96.

Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27(1):10–28.

Lederman LC. Debriefing: toward a systematic assessment of theory and practice. Simul Gaming. 1992;23(2):145–60.

Savoldelli GL, Naik VN, Park J, Joo HS, Chow R, Hamstra SJ. Value of debriefing during simulated crisis management: oral versus video-assisted oral feedback. Anesthesiology. 2006;105(2):279–85.

Tannenbaum SI, Cerasoli CP. Do team and individual debriefs enhance performance? A meta-analysis. Hum Factors. 2013;55(1):231–45.

Arora S, Miskovic D, Hull L, Moorthy K, Aggarwal R, Johannsson H, et al. Self vs expert assessment of technical and non-technical skills in high fidelity simulation. Am J Surg. 2011;202(4):500–6.

Burns CL. Using debriefing and feedback in simulation to improve participant performance: an educator’s perspective. Int J Med Educ. 2015;6:118–20.

Roussin CJ, Weinstock P. SimZones: an organizational innovation for simulation programs and centers. Acad Med. 2017;92(8):1114–20.

Gordon M, Farnan J, Grafton-Clarke C, Ahmed R, Gurbutt D, McLachlan J, et al. Non-technical skills assessments in undergraduate medical education: a focused BEME systematic review: BEME Guide no. 54. Med Teach. 2019:41(7)732-45.

Salas E, Reyes DL, Woods AL. The assessment of team performance: observations and needs. Innovative Assessment of Collaboration. Methodology of Educational Measurement and Assessment; 2017. p. 21–36.

Hull L, Arora S, Stefanidis D, Sevdalis N. Facilitating the implementation of the American College of Surgeons/Association of Program Directors in Surgery phase III skills curriculum: training faculty in the assessment of team skills. Am J Surg. 2015;210(5):933-41 e2.

Baker DP, Dismukes RK. A framework for understanding crew performance assessment issues. Int J Aviat Psychol. 2002;12(3):205–22.

Boet S, Bould MD, Layat Burn C, Reeves S. Twelve tips for a successful interprofessional team-based high-fidelity simulation education session. Med Teach. 2014;36(10):853–7.

Lindqvist SM, Reeves S. Facilitators’ perceptions of delivering interprofessional education: a qualitative study. Med Teach. 2007;29(4):403–5.

Russ S, Hull L, Rout S, Vincent C, Darzi A, Sevdalis N. Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training. Ann Surg. 2012;255(4):804–9.

Rall M, Gaba D, Howard SK, Dieckmann P. Human performance and patient safety. In: Gropper M, Eriksson L, Fleisher L, Wiener-Kronish J, Cohen N, Leslie K, editors. Miller’s Anesthesia; 2010.

Rudolph JW, Simon R, Raemer DB, Eppich WJ. Debriefing as formative assessment: closing performance gaps in medical education. Acad Emerg Med. 2008;15(11):1010–6.

Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefing with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiol Clin. 2007;25(2):361–76.

Cheng A, Morse KJ, Rudolph J, Arab AA, Runnacles J, Eppich W. Learner-centered debriefing for health care simulation education: lessons for faculty development. Simul Healthc. 2016;11(1):32–40.

Fey MK, Roussin CJ, Rudolph JW, Morse KJ, Palaganas JC, Szyld D. Teaching, coaching, or debriefing with good judgment: a roadmap for implementing ‘with good judgment’ across the SimZones. Adv Simul (Lond). 2022;7(1):39.

Rosen MA, Weaver SJ, Lazzara EH, Salas E, Wu T, Silvestri S, et al. Tools for evaluating team performance in simulation-based training. J Emerg Trauma Shock. 2010;3(4):353–9.

Eppich W, Nannicelli AP, Seivert NP, Sohn MW, Rozenfeld R, Woods DM, et al. A rater training protocol to assess team performance. J Contin Educ Health Prof. 2015;35(2):83–90.

Feldman M, Lazzara EH, Vanderbilt AA, DiazGranados D. Rater training to support high-stakes simulation-based assessments. J Contin Educ Health Prof. 2012;32(4):279–86.

Gingerich A, Kogan J, Yeates P, Govaerts M, Holmboe E. Seeing the ‘black box’ differently: assessor cognition from three research perspectives. Med Educ. 2014;48(11):1055–68.

van der Vleuten CP, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ. 2005;39(3):309–17.

Govaerts MJ, Schuwirth LW, Van der Vleuten CP, Muijtjens AM. Workplace-based assessment: effects of rater expertise. Adv Health Sci Educ Theory Pract. 2011;16(2):151–65.

Weller J, Shulruf B, Torrie J, Frengley R, Boyd M, Paul A, et al. Validation of a measurement tool for self-assessment of teamwork in intensive care. Br J Anaesth. 2013;111(3):460–7.

Cheng A, Palaganas J, Eppich W, Rudolph J, Robinson T, Grant V. Co-debriefing for simulation-based education: a primer for facilitators. Simul Healthc. 2015;10(2):69–75.

Jenkins B. Training and assessment of non-technical skills in the operating theatre: where next? Anaesthesia. 2015;70:893–906.

Geraghty A, Paterson-Brown S. Non-technical skills for surgeons (NOTSS). Surg Infect (Larchmt). 2020;38(10):612–6.

Flin R, Mitchell L, McLeod B. Non-technical skills of the scrub practitioner: the SPLINTS system. ORNAC J. 2014;32(3):33–8.

Flin R, Patey R. Non-technical skills for anaesthetists: developing and applying ANTS. Best Pract Res Clin Anaesthesiol. 2011;25(2):215–27.

Guise JM, Deering SH, Kanki BG, Osterweil P, Li H, Mori M, et al. Validation of a tool to measure and promote clinical teamwork. Simul Healthc. 2008;3(4):217–23.

Kolbe M, Burtscher MJ, Manser T. Co-ACT–a framework for observing coordination behaviour in acute care teams. BMJ Qual Saf. 2013;22(7):596–605.

Frei-Landau R, Levin O. Simulation-based learning in teacher education: using Maslow’s Hierarchy of needs to conceptualize instructors’ needs. Front Psychol. 2023;14:1149576.

Hull L, Arora S, Symons NR, Jalil R, Darzi A, Vincent C, et al. Training faculty in nontechnical skill assessment: national guidelines on program requirements. Ann Surg. 2013;258(2):370–5.

Eva KW, Bordage G, Campbell C, Galbraith R, Ginsburg S, Holmboe E, et al. Towards a program of assessment for health professionals: from training into practice. Adv Health Sci Educ Theory Pract. 2016;21(4):897–913.

Hull L, Sevdalis N. Advances in teaching and assessing nontechnical skills. Surg Clin North Am. 2015;95(4):869–84.

Thông tin
Thông tin xuất bản

Advances in Simulation

Thông tin tác giả