Summative Assessment in Medicine: The Promise of Simulation for High‐stakes Evaluation

Academic Emergency Medicine - Tập 15 Số 11 - Trang 1017-1024 - 2008
John R. Boulet1
1From the Foundation for Advancement of International Medical Education and Research, Philadelphia, PA.

Tóm tắt

AbstractThroughout their careers, physicians are exposed to a wide array of assessments, including those aimed at evaluating knowledge, clinical skills, and clinical decision‐making. While many of these assessments are used as part of formative evaluation activities, others are employed to establish competence and, as a byproduct, to promote patient safety. In the past 10 years, simulations have been successfully incorporated in a number of high‐stakes physician certification and licensure exams. In developing these simulation‐based assessments, testing organizations were able to promote novel test administration protocols, build enhanced assessment rubrics, advance sophisticated scoring and equating algorithms, and promote innovative standard‐setting methods. Moreover, numerous studies have been conducted to identify potential threats to the validity of test score interpretations. As simulation technology expands and new simulators are invented, this groundbreaking work can serve as a basis for organizations to build or expand their summative assessment activities. Although there will continue to be logistical and psychometric problems, many of which will be specialty‐ or simulator‐specific, past experience with performance‐based assessments suggests that most challenges can be addressed through focused research. Simulation, whether it involves standardized patients (SPs), computerized case management scenarios, part‐task trainers, electromechanical mannequins, or a combination of these methods, holds great promise for high‐stakes assessment.

Từ khóa


Tài liệu tham khảo

HubbardJ LevitE.The National Board of Medical Examiners: The First Seventy Years. Philadelphia PA:National Board of Medical Examiners 1985.

10.1056/NEJM196506242722505

10.1056/NEJMra054784

10.1097/01.ccx.0000132654.52131.32

10.1016/j.atc.2006.11.004

10.1111/j.1365-2929.2006.02394.x

10.1176/appi.ap.30.6.505

10.1136/qshc.2004.009886

10.1136/emj.2005.030544

Maran NJ, 2003, Low‐ to high‐fidelity simulation‐‐a continuum of medical education?, Med Educ., 37, 22, 10.1046/j.1365-2923.37.s1.9.x

10.1097/00001888-199810000-00057

10.1016/j.anclin.2007.03.003

10.1080/01421590500315139

10.1213/01.ane.0000194934.34552.ab

10.1016/j.anclin.2007.03.002

Medical News Today.Medical Simulation Corporation and the American Board of Internal Medicine Introduce New Simulation for Physician Assessment and Evaluation. Med News Today 2‐11‐2008. Available at:http://www.medicalnewstoday.com/articles/96850.php. Accessed Jul 16 2008.

10.1136/qshc.2004.010025

Melnick DE, 2002, Medical licensing examinations in the United States, J Dent Educ., 66, 595, 10.1002/j.0022-0337.2002.66.5.tb03537.x

10.1097/00001888-200210001-00029

10.1002/chp.1340230310

10.1001/jama.283.13.1748

Gimpel JR, 2003, Evaluating the clinical skills of osteopathic medical students, J Am Osteopath Assoc., 103, 267

10.1097/01.ACM.0000236513.54577.b5

10.1097/00001888-200510001-00010

10.1080/01421590500046924

10.1097/00005373-200107000-00003

Hawkins RE, 2008, Practical Guide to the Evaluation of Clinical Competence, 102

Boulet JR, 2004, Simulations in Critical Care Education and Beyond, 119

Boulet JR, 2003, Using National Medical Care Survey data to validate examination content on a performance‐based clinical skills assessment for osteopathic physicians, J Am Osteopath Assoc., 103, 225

10.1023/A:1022639521218

10.1016/j.amjmed.2005.10.036

10.1111/j.1365-2929.2004.01932.x

10.1213/01.ANE.0000090152.91261.D9

10.1148/radiol.2411062581

10.1001/jama.282.9.861

10.1111/j.1365-2923.2007.02799.x

10.1197/aemj.9.11.1310

Accreditation Council for Graduate Medical Education.ACGME Outcome Project.2004. Available at:http://www.acgme.org/outcome. Accessed Jul 16 2008.

10.1001/jama.287.2.226

10.1197/j.aem.2004.05.037

10.1046/j.1365-2923.37.s1.8.x

Ziv A, 2000, Proceedings of the Eighth Ottawa Conference on Medical Education and Assessment, 99

National Center for Health Statistics.National Hospital Ambulatory Medical Care Survey (NHAMCS). Hyattsville MD. Available at:http://www.cdc.gov/nchs/nhamcs.htm. Accessed Jul 16 2008.

10.1111/j.1365-2044.2006.04902.x

10.1197/j.aem.2006.11.021

10.1197/aemj.9.11.1295

10.1097/SIH.0b013e3181626276

10.1353/pbm.2008.0004

10.1016/j.amjmed.2007.06.017

10.1353/pbm.2008.0003

10.1080/01421590500126296

10.1016/j.ambp.2006.12.005

10.1097/01.anes.0000286926.01083.9d

10.1097/00000542-200608000-00008

10.1097/00001888-200110000-00016

10.1097/01.anes.0000265149.94190.04

10.1097/00001888-200310001-00015

10.1097/00000542-200411000-00007

10.1007/BF03016633

10.1093/bja/aeh081

10.1007/s10459-006-9024-4

10.1542/peds.2007-0029

10.1080/01421590701513698

10.1016/S1553-7250(05)31025-7

10.1097/ACM.0b013e318141f40a

10.1093/bja/aeg002

10.1023/B:AHSE.0000012216.39378.15

10.1023/A:1009862220473

10.1177/1062860606296143

10.1056/NEJMe068212

10.1111/j.1365-2923.1991.tb00036.x

10.1197/j.aem.2006.01.030

10.1542/peds.2005-1259

Boulet JR, 2005, Encyclopedia of Statistics in Behavioral Science, 704

Thompson B, 2003, Score Reliability: Contemporary Thinking on Reliability Issues, 43, 10.4135/9781412985789

10.2307/1177222

10.1046/j.1365-2923.2003.01594.x

10.1111/j.1745-3984.2001.tb01130.x

Clauser BE, 2008, Practical Guide to the Evaluation of Clinical Competence, 10

10.1111/j.1365-2923.2004.01755.x

Grand’Maison P, 1996, Content validity of the Quebec licensing examination (OSCE). Assessed by practising physicians, Can Fam Phys., 42, 254

Berkenstadt H, 2006, The validation process of incorporating simulation‐based accreditation into the anesthesiology Israeli national board exams, Isr Med Assoc J., 8, 728

10.1097/00001888-200105000-00019

10.1089/lap.2006.0221

10.1097/00000542-200212000-00015

10.1016/j.amjsurg.2006.08.066

10.1097/SIH.0b013e31816e39e2

10.1056/NEJMra054785

10.1001/jama.298.9.993

10.1111/j.1365-2923.2007.02953.x

10.7326/0003-4819-144-2-200601170-00008

10.1097/00000539-200201000-00028

10.1007/s11606-007-0283-4

10.1097/MCC.0b013e3282f1bb32

10.1097/TA.0b013e31816275b0

10.1377/hlthaff.26.6.1712

Thông tin
Thông tin xuất bản

Academic Emergency Medicine

Tập 15 Số 11

1017-1024

Thông tin tác giả