Development of a Tool to Assess Medical Oral Language Proficiency

Academic Medicine - Tập 98 Số 4 - Trang 480-490 - 2023
Lisa C. Diamond1,2, Steven E. Gregorich3,4, Leah Karliner5,6, Javier González7, Cristina Pérez-Cordón8, Reniell X. Iñiguez9,10, José Alberto Figueroa11,12, Karen Izquierdo13,14, Pilar Ortega15,16
1L.C. Diamond
2is associate attending physician, Immigrant Health and Cancer Disparities Service, Hospital Medicine Service, Departments of Medicine and Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, New York; ORCID:.
3S. Gregorich
4is professor emeritus, Division of General Internal Medicine, Department of Medicine, University of California, San Francisco, San Francisco, California.
5L. Karliner
6is professor, Division of General Internal Medicine, Center for Aging in Diverse Communities, Multiethnic Health Equity Research Center, Department of Medicine, University of California, San Francisco, San Francisco, California; ORCID:.
7is program manager, Language Initiatives Program, Immigrant Health and Cancer Disparities Service, Department of Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, New York.
8is a language assessment specialist, Language and Communication Training Unit, United Nations Headquarters, New York, New York.
9R. Iniguez
10is a medical student, University of Illinois College of Medicine, Chicago, Illinois.
11J.A. Figueroa
12is a medical student, Northwestern University Feinberg School of Medicine, Chicago, Illinois.
13K. Izquierdo
14is a medical student, College of Medicine, State University of New York at Downstate Health Sciences University, Brooklyn, New York.
15P. Ortega
16is clinical assistant professor, Departments of Medical Education and Emergency Medicine, University of Illinois College of Medicine, Chicago, Illinois; ORCID:.

Tóm tắt

Purpose To communicate with linguistically diverse patients, medical students and physicians often use their non-English-language skills. However, there is no standard protocol to determine whether those skills are adequate before patient care. This causes many physicians, institutions, educators, and learners to forgo non-English-language proficiency assessment altogether. The purpose of this study is to report on the development, refinement, and interrater reliability of the Physician Oral Language Observation Matrix (POLOM), a rater-based tool assessing 6 language skill categories observed during clinical interactions: comprehension, fluency/fluidity, vocabulary, pronunciation, grammar, and communication. This study focused on the use of the POLOM in Spanish interactions. Method The authors adapted an existing language observation tool for use in clinical settings, creating the preliminary POLOM. Next, they iteratively refined the tool from April to July 2021 using videorecorded medical student-standardized patient encounters from a U.S.-based medical Spanish program. In each refinement iteration, 4 bilingual raters (2 physicians and 2 linguists) independently rated 3 to 6 encounters and convened to discuss ratings with the goals of improving instrument instructions, descriptors, and subsequent rater agreement. Using the final POLOM, raters independently rated 50 videos in rotating interdisciplinary pairs. Generalizability theory was applied to estimate reliability via interrater agreement (dependability) coefficients (range 0–1) for each POLOM category and the total score. Results POLOM total score dependability equaled 0.927 (single rater) and 0.962 (averaged across 2 raters). The highest mean score was observed for the comprehension category (4.15; range 1–5) while the lowest was for communication (3.01; range 1–5). Conclusions Raters achieved a high level of agreement on POLOM assessments of students’ medical oral Spanish proficiency. The POLOM is the first assessment tool that provides examinees and instructors with both a holistic and detailed review of clinician non-English oral language skills as contextualized for patient care.

Từ khóa


Tài liệu tham khảo

Ortega, 2022, Rethinking the term “limited English proficiency” to improve language-appropriate healthcare for all., J Immigr Minor Health, 24, 799, 10.1007/s10903-021-01257-w

2000, Improving access to services for persons with limited English proficiency. Executive Order 13166., Fed Regist, 65, 50119

2016, Nondiscrimination in health programs and activities: Final rule., Fed Regist, 81

Weech-Maldonado, 2004, Health plan effects on patient assessments of Medicaid managed care among racial/ethnic minorities., J Gen Intern Med, 19, 136, 10.1111/j.1525-1497.2004.30235.x

Ngo-Metzger, 2007, Providing high-quality care for limited English proficient patients: The importance of language concordance and interpreter use., J Gen Intern Med, 22, 324, 10.1007/s11606-007-0340-z

Solis, 1990, Acculturation, access to care, and use of preventive services by Hispanics: Findings from HHANES 1982-84., Am J Public Health, 80, 11–19, 10.2105/AJPH.80.Suppl.11

Divi, 2007, Language proficiency and adverse events in US hospitals: A pilot study., Int J Qual Health Care, 19, 60, 10.1093/intqhc/mzl069

Diamond, 2014, Non–English-language proficiency of applicants to US residency programs., JAMA, 312, 2405, 10.1001/jama.2014.15444

Vela, 2016, Medical students’ experiences and perspectives on interpreting for LEP patients at two US medical schools., J Racial Ethn Health Disparities, 3, 245, 10.1007/s40615-015-0134-7

Diamond, 2009, Getting by: Underuse of interpreters by resident physicians., J Gen Intern Med, 24, 256, 10.1007/s11606-008-0875-7

Prince, 1995, Teaching Spanish to emergency medicine residents., Acad Emerg Med, 2, 32, 10.1111/j.1553-2712.1995.tb03076.x

Ortega, 2021, Medical Spanish in US medical schools: A national survey to examine existing programs., J Gen Intern Med, 36, 2724, 10.1007/s11606-021-06735-3

Diamond, 2012, “Does this doctor speak my language?” Improving the characterization of physician non-English language skills., Health Serv Res, 47, 556, 10.1111/j.1475-6773.2011.01338.x

Diamond, 2014, Relationship between self-assessed and tested non-English-language proficiency among primary care providers., Med Care, 52, 435, 10.1097/MLR.0000000000000102

Ortega, 2020, Medical Spanish standardization in U.S. medical schools: Consensus statement from a multidisciplinary expert panel., Acad Med, 95, 22, 10.1097/ACM.0000000000002917

Tang, 2011, The Kaiser Permanente Clinician Cultural and Linguistic Assessment Initiative: Research and development in patient-provider language concordance., Am J Public Health, 101, 205, 10.2105/AJPH.2009.177055

Ortega, 2017, Evaluation of a medical Spanish elective for senior medical students: Improving outcomes through OSCE assessments., Med Sci Educ, 27, 329, 10.1007/s40670-017-0405-5

Dennis, 2019, The student oral Language observation matrix: A psychometric study with preschoolers., Assess Eff Interv, 45, 65

McConkey Robbins, 2004, Bilingual oral language proficiency in children with cochlear implants., Arch Otolaryngol Head Neck Surg, 130, 644, 10.1001/archotol.130.5.644

Albanese, 2011, Commentary: Measurement and interpretation challenges in comparing student performance outcomes from different medical schools., Acad Med, 86, 1073, 10.1097/ACM.0b013e318226340c

Augustin, 2014, How to learn effectively in medical school: Test yourself, learn actively, and repeat in intervals., Yale J Biol Med, 87, 207

Karpicke, 2009, Metacognitive strategies in student learning: Do students practise retrieval when they study on their own?., Memory, 17, 471, 10.1080/09658210802647009

Shin, 2021, Scoping review of textbooks for medical Spanish education., Med Sci Educ, 31, 1519, 10.1007/s40670-021-01333-8

Ortega, 2018, Spanish language concordance in U.S. medical care: A multifaceted challenge and call to action., Acad Med, 93, 1276, 10.1097/ACM.0000000000002307

Diamond, 2019, A systematic review of the impact of patient-physician non-English language concordance on quality of care and outcomes., J Gen Intern Med, 34, 1591, 10.1007/s11606-019-04847-5

Lion, 2013, Clinical Spanish use and language proficiency testing among pediatric residents., Acad Med, 88, 1478, 10.1097/ACM.0b013e3182a2e30d

Diamond, 2010, Do hospitals measure up to the national culturally and linguistically appropriate services standards?., Med Care, 48, 1080, 10.1097/MLR.0b013e3181f380bc

Thông tin
Thông tin xuất bản

Academic Medicine

Tập 98 Số 4

480-490

Thông tin tác giả