A Novel Simulation to Assess Residents' Utilization of a Medical Interpreter
By the end of this activity, faculty will be able to:
Evaluate a learner's ability to employ a medical interpreter in gathering a history.
Evaluate a learner's ability to identify and manage postpartum eclampsia.
Evaluate a learner's ability to communicate effectively with a family member and members of the health care team.
Non-English-speaking patients face significant barriers to care and have a lower satisfaction with their physician-patient relationship.1–8 However, the use of a qualified medical interpreter (MI) improves patient and provider satisfaction and is mandated for communication when the provider is not fluent in the patient's preferred language.9–14 As a result, physicians must be facile in working with an MI in the clinical setting.
We performed a literature review of MEDLINE (using PubMed, keywords medical interpreter simulation and medical interpreter resident) and MedEdPORTAL (keywords medical interpreter), which yielded several curricula focused on the use of MIs. Many of the prior curricula were geared toward medical student education, and although some incorporated standardized patients, none utilized simulation to assess learner competence in the use of interpreters.15–20 We found two curricula specifically designed for residents, but neither included a significant assessment component.21,22
The Accreditation Council for Graduate Medical Education's (ACGME's) Clinical Learning Environment Review process mandates education on reducing health care disparities. Despite this mandate and the prior published curricula, a survey of residents in pediatrics residency programs found that 54% never received formal education in the use of MIs. Additionally, there is a dearth of tools for assessing resident competence in this skill. Thus, we developed a low-stakes, formative simulation assessment tool to address health care of the non-English-speaking patient.
To evaluate and improve emergency medicine residents' skills in working with an MI, we developed a simulation scenario to allow for multisource assessment of their ability to communicate with a Spanish-speaking family member in an emergency situation. We chose the use of simulation over other modalities (e.g., direct assessment of patient encounters) for the ability to standardize the encounter for all residents and for the ability to obtain a multisource standardized assessment of the residents' performance. To implement this scenario, we adapted a case we had previously used that involved a patient with eclamptic seizures and made a certified MI available to the residents during the case.23
Recognizing the need to assess our residents' ability to use MIs according to ACGME Emergency Medicine Milestone 22, Patient Centered Communication,24 we adapted a previously existing case of eclamptic seizures and incorporated the need for an MI by making the patient and her family non–native English speaking. The case was developed by the program directors and departmental simulation director in conjunction with the program's education committee. Assistance was provided by the university's simulation staff and the MIs who ultimately participated in the case.
This case was implemented as part of our emergency medicine residency's semiannual formative simulation assessment program. As part of this program, each resident participated in two simulation assessment cases every 6 months. All cases were calibrated to be challenging for a senior resident to complete successfully. Results were used in conjunction with other evaluative data to identify residents in need of additional support or remediation and to assess resident competency. Validity evidence for the assessment tool was compiled before delivering the data to the clinical competency committee for use in resident milestone-level assignments.25 The assessment tool and simulation case were reviewed by multiple board-certified emergency medicine faculty prior to pilot testing, with adjustments made to incorporate feedback, lending content validity.
Equipment used in the simulation included the following:
High-fidelity mannequin (female, penis removed, episiotomy scar on perineum).
Noninvasive blood pressure cuff.
Three-lead EKG and pulse oximetry monitor.
IV, IV pole, normal saline bag and tubing, various syringes to simulate medication.
Simulated midazolam, magnesium sulfate, phenytoin or other antiepileptic drugs, labetalol or other antihypertensive drugs, drugs for rapid-sequence intubation.
Airway supplies including nasal cannula, nonrebreather mask, bag-valve-mask device, Mac and Miller blades, endotracheal tubes, waveform capnography.
Television screen or computer monitor for displaying imaging and laboratory results (see Appendix B).
Personnel in the simulation included the following:
Faculty observer and simulation controller.
Simulated participant: nurse.
Simulated participant: patient's husband (could substitute a concerned sister/mother/partner if female), must be fluent in Spanish.
Certified Spanish MI.
The case took place in a high-fidelity simulation laboratory. Briefly, the case involved a Spanish-speaking female of childbearing age who had given birth the prior week and then developed eclamptic seizures. The full case script is provided in Appendix A. Each case took approximately 15 minutes, and each learner completed the case once. Approximately 20 cases were completed per room per day.
Setup included briefing the MI, the simulated participant who played the patient's Spanish-speaking husband or sister (see Appendix B), the nurse simulated participant, and the faculty observer. Upon arrival at the simulation area but prior to entry into the room, the learner was briefed on standard rules of our institution's simulation laboratory. The learner was then provided with the case stem (see Appendix A) before entering the simulation room. No additional review material was given to the learner prior to the evaluation.
The MIs were recruited from the local children's hospital interpreter pool, as well as from a local volunteer group. All were certified Spanish MIs. The interpreter department director was contacted, and the voluntary opportunity was offered to all certified Spanish MIs. The MIs were oriented to the case on the morning of the assessment. Further preassessment training of MIs was determined to be unnecessary, as they were asked to participate in the case in their typical professional role. Prior to the formal start day of the assessments, the MIs reviewed the script to familiarize themselves with the flow of the simulation, and they participated in the final walk-through to ensure their familiarity with the case. The simulated family member and nurse simulated participant were recruited from our medical school simulation center's pool of standardized actors. The family member was chosen for his bilingual fluency, realistic portrayals in other simulation activities, and experience. Specific attention was placed on his demeanor, as it was important for him to maintain his professionalism and remain in character despite potentially negative interactions during the simulation. The nurse simulated participant was chosen for her critical care background, experience with emergency medicine simulation, and ability to provide realistic portrayals as an emergency medicine nurse.
Both simulated participants were given copies of the case to be memorized before training began. Training consisted of a tabletop walk-through with both simulated participants, faculty leaders, and simulation operations staff. Following this, the case was run several times in the simulation room with faculty portraying the residents. These sessions took 2–3 hours and occurred prior to the first day of formal assessments. A final walk-through of the case was completed on the morning of the first day of assessments as a final check to ensure all participants were familiar and comfortable with their roles.
Faculty observers were recruited from within the department. The 15 faculty who participated in rating this case were provided with the case and assessment tool, as well as general information on the structure of the assessment, prior to the testing day. Upon arrival, they were again briefed on the case by the faculty leads, and any questions were answered. Faculty observers were not involved in running the case; they performed only ratings and debriefing.
Initially, only the nurse simulated participant and simulated patient were in the room with the learner; the simulated family member and MI were out of sight in the control room. The learner received the Emergency Medical Services report verbally from the simulation operator but was not made privy to the patient's recent obstetric history. The patient was post-ictal and unable to provide a history in any language, only moaning during examination of her abdomen. Shortly after the case began, the patient seized again, prompting the learner to administer an antiepileptic. During the seizure, the patient's Spanish-speaking husband arrived in the room, was distraught, and immediately sought information from the learner on his wife's status. As the husband demonstrated concern and requested information in Spanish, the learner had to determine how to communicate effectively with him while continuing to medically manage the seizing patient.
If the learner requested interpretive services, the MI arrived in the room. The learner could then use the MI to obtain history from the patient's husband that strongly suggested eclampsia as the cause for her seizures. Laboratory, radiology, and other diagnostic results are detailed in Appendix C. Ideally, the learner should then have ordered magnesium and treated the patient's elevated blood pressure. Otherwise, she would continue to seize and would develop status epilepticus despite the administration of other antiepileptic drugs. The gynecologist, if called, would initially try to avoid immediate consultation by stating he or she was in the OR. However, if the learner was insistent, the specialist would agree to respond immediately. The case ended after the gynecologist accepted and admitted the patient to the ICU, magnesium was given, hypertension was appropriately treated, and the patient's husband was updated with her diagnosis and plan. Otherwise, the case ended after 15 minutes.
All residents were evaluated on observable actions that were anchored to the language of the Emergency Medicine Milestones.24 The learner was assessed by both the faculty observer and the MI who participated in the case. The MI's assessment (Appendix D) focused on the resident's interpersonal communication skills, including whether the resident requested an MI or was already fluent in Spanish, utilized the MI properly by maintaining eye contact with and speaking directly to the family, and used lay terminology in a culturally sensitive manner. Although we did not utilize the simulated family member for feedback, the MI assessment form could easily be adapted for use by this person.
The faculty observer's assessment (Appendix D) focused on the broader patient care, including whether the resident performed a primary resuscitation assessment, completed all important elements of the history and physical examination, ordered appropriate testing, appropriately recognized and treated the eclamptic seizure, and consulted the gynecologist on call. The faculty observer also evaluated the resident's communication skills. Critical actions were created based on the ACGME Emergency Medicine Milestones.24
After the case, the learner was debriefed in a separate classroom by the faculty observer. The observer reviewed the case and the learner's actions and clarified any ambiguous actions that had occurred during the simulation. The observer also reviewed the critical actions and provided verbal feedback on the learner's performance. The correct diagnosis and treatment were discussed, and the learner was given the opportunity to explain his or her thought process during the case. Finally, the appropriate use of an MI was discussed, including the need to use short phrases, pause frequently for interpretation, and speak directly to the patient (or family member).
Probing questions during the debriefing included the following:
What is the differential diagnosis for a patient presenting with seizures?
What are the first- and second-line treatments for status epilepticus?
What is the definition of preeclampsia and eclampsia?
How are eclamptic seizures treated?
Tell me how you typically work with a medical interpreter with a non-English-speaking patient.
Appendix E contains a tip sheet for the use of an MI clinically. Although we did not utilize this appendix with our learners, we created it as a resource for educators to use when implementing this case. Additionally, the cited work serves as a good overview of the diagnosis and treatment of eclampsia for both faculty reference and learner review.26
The simulation assessment was performed with 60 emergency medicine residents, supervised by emergency medicine faculty, in fall 2016. Only four residents did not request an MI, two of whom were native speakers. Both native speakers were evaluated for their ability to communicate clearly with the patient and family and were given an evaluation of “unable to assess” for their use of the MI. Common areas for improvement were identified regarding the use of lay terminology, as well as maintaining proper eye contact and body positioning, when using an MI. Compared to the prior version of the case that utilized the same eclamptic seizure scenario but did not include the language barrier element, residents received lower overall scores on the new version. Overall scores on the modified case were a median of 10 out of 27 (odds ratio = 0.43; 95% confidence interval, 0.18–0.99) compared to a median score of 12 out of 24 achieved on the prior version, with a significant difference between the percentage of total items correct for the two groups (p = .02). This suggests that the inclusion of a language barrier created a more challenging case for the residents.
Median scores on the assessment were noted to decrease by PGY subgroup (PGY 3: 20.5; PGY 2: 19.5; PGY 1: 15.0), showing a split between junior and senior residents and lending evidence for construct validity of this simulation as an assessment tool. Response process validity was addressed through extensive simulation operator and standardized patient training, as well as instructing raters (interpreter and faculty) on the proper use of the assessment tool. Scores for the subcompetency Interpersonal and Communication Skills 1, which addresses the use of interpreters in communication, were similar between the interpreter and the faculty raters (median score of 5 overall and per each class, maximum possible score of 6).
To conduct a formative assessment of emergency medicine residents on the use of MIs, we adapted a previously used case of eclamptic seizures to include a non-English-speaking patient and relative. Overall, residents scored lower on this version of the assessment than on the prior version that omitted the language barrier, and areas of improvement for the learners included adopting the standard mechanics of using an interpreter.
The process of adding an MI to a previously developed case is challenging but would be possible if an instructor wishes to introduce a language barrier to a different simulation scenario. On reflection, we believe that a moderately difficult case in which key elements of the history must be obtained via an interpreter would allow for maximal emphasis on the importance of proper interpreter use. We also added an element of emotional distress when the Spanish-speaking family member entered the room in order to challenge the learner to manage emotions while obtaining a history with the MI. This element could be omitted in a different case if the instructor wishes to limit the added challenge to the language barrier alone. In any case in which a language barrier is used, it is important to allocate sufficient time for the learner to recognize the need for interpretive services, for the MI to appear in the room, and for the history to be obtained using a third person. Finally, a case could be similarly modified to use an interpreter over the phone. In such a case, the trained MI could be positioned in the control room, with a double-handset phone, tablet computer, or other similar device used for communication.
There were several challenges that we did not anticipate during implementation of the case. During the debriefings, it became clear that some residents had not expected an MI to be available, with some not requesting an MI and later stating they otherwise would have. As a result, we suggest that the door prompt or nurse script include a cue that MI services are available upon request or that all ancillary services typically available at the hospital are available.
Additionally, several of our Spanish-speaking residents appropriately chose to conduct the interview without the MI. If an instructor wishes to prevent this from occurring in the future, the instructor could modify the case to utilize another language that is not known to any of the learners. This, of course, would require an assessment of the languages in which learners are fluent. It could also lead to additional difficulty in finding interpreters and patient actors who are comfortable working in the chosen language.
There are several limitations to this assessment as it currently exists. First, we designed the assessment for formative purposes, with an emphasis on learner education. As such, we did not control for variability between MI and faculty observations. Additionally, to facilitate participation by MIs, their orientation to the case and assessment system was minimal and could have resulted in limited interrater reliability. Although we believe that a similar case could be used for high-stakes assessment, several preimplementation changes would be required. These include further validation of the scoring system and the creation of a standardized training mechanism for evaluators. Furthermore, we designed this case specifically for emergency medicine residents, and the underlying diagnosis may be too difficult or esoteric for residents in unrelated specialties.
Following the implementation of this case, we found that many of our residents did not know the basic mechanics of using an MI. This anecdotal finding correlates with the literature finding that fewer than half of residents have received formal training in the use of MIs, and it represents an area of opportunity to improve the care of vulnerable patient populations.
Although this case was created specifically for the training of emergency medicine residents, it could be adapted for use by other specialties, such as obstetrics and family medicine, as well as for use in the training of medical students. Other specialties could utilize a similarly designed case with a disease process specific to their area of expertise. We believe that the pathology underlying the case is less important because the primary focus of the case is on the appropriate use of an MI.
The use of MIs is an integral part of health care practice in the United States. We present a simulation case that can assess learners' use of MIs and allow for an educational intervention that may improve the care of non-English-speaking patients.
Flores G. The impact of medical interpreter services on the quality of health care: a systematic review. Med Care Res Rev. 2005;62(3):255–299. https://doi.org/10.1177/1077558705275416Google Scholar
Flower KB, Skinner AC, Yin HS, et al.Satisfaction with communication in primary care for Spanish-speaking and English-speaking parents. Acad Pediatr. 2017;17(4):416–423. https://doi.org/10.1016/j.acap.2017.01.005Google Scholar
Menendez ME, Loeffler M, Ring D. Patient satisfaction in an outpatient hand surgery office: a comparison of English- and Spanish-speaking patients. Qual Manag Health Care. 2015;24(4):183–189. https://doi.org/info:doi/10.1097/QMH.0000000000000074Google Scholar
Ramirez D, Engel KG, Tang TS. Language interpreter utilization in the emergency department setting: a clinical review. J Health Care Poor Underserved. 2008;19(2):352–362. https://doi.org/10.1353/hpu.0.0019Google Scholar
Baker DW, Hayes R, Fortier JP. Interpreter use and satisfaction with interpersonal aspects of care for Spanish-speaking patients. Med Care. 1998;36(10):1461–1470. https://doi.org/10.1097/00005650-199810000-00004Google Scholar
Brice JH, Travers D, Cowden CS, Young MD, Sanhueza A, Dunston Y. Health literacy among Spanish-speaking patients in the emergency department. J Natl Med Assoc. 2008;100(11):1326–1332. https://doi.org/10.1016/S0027-9684(15)31512-1Google Scholar
Villani J, Mortensen K. Decomposing the gap in satisfaction with provider communication between English- and Spanish-speaking Hispanic patients. J Immigr Minor Health. 2014;16(2):195–203. https://doi.org/10.1007/s10903-012-9733-0Google Scholar
Callahan E, Garcia E, Rehm J. Talk louder? Communicating with your Spanish speaking patients. MedEdPORTAL. 2011;7:8427. https://doi.org/10.15766/mep_2374-8265.8427Google Scholar
Bagchi AD, Dale S, Verbitsky-Savitz N, Andrecheck S, Zavotsky K, Eisenstein R. Examining effectiveness of medical interpreters in emergency departments for Spanish-speaking patients with limited English proficiency: results of a randomized controlled trial. Ann Emerg Med. 2011;57(3):248–256.e4. https://doi.org/10.1016/j.annemergmed.2010.05.032Google Scholar
Kuo D, Fagan MJ. Satisfaction with methods of Spanish interpretation in an ambulatory care clinic. J Gen Intern Med. 1999;14(9):547–550. https://doi.org/10.1046/j.1525-1497.1999.07258.xGoogle Scholar
Chen AH, Youdelman MK, Brooks J. The legal framework for language access in healthcare settings: Title VI and beyond. J Gen Intern Med. 2007;22(suppl 2):362–367. https://doi.org/10.1007/s11606-007-0366-2Google Scholar
- 12. Title VI, Prohibition Against National Origin Discrimination Affecting Limited English Proficient Persons. National Archives website. https://www.archives.gov/eeo/laws/title-vi.html. Published August15, 2016. Accessed September 3, 2018.Google Scholar
Karliner LS, Jacobs EA, Chen AH, Mutha S. Do professional interpreters improve clinical care for patients with limited English proficiency? A systematic review of the literature. Health Serv Res. 2007;42(2):727–754. https://doi.org/10.1111/j.1475-6773.2006.00629.xGoogle Scholar
Silva MD, Genoff M, Zaballa A, et al.Interpreting at the end of life: a systematic review of the impact of interpreters on the delivery of palliative care services to cancer patients with limited English proficiency. J Pain Symptom Manage. 2016;51(3):569–580. https://doi.org/10.1016/j.jpainsymman.2015.10.011Google Scholar
Lie D. Interpreter cases for cultural competency instruction. MedEdPORTAL. 2006;2:205. https://doi.org/10.15766/mep_2374-8265.205Google Scholar
Lie D, Bereknyei S, Braddock CH III, Encinas J, Ahearn S, Boker JR. Assessing medical students' skills in working with interpreters during patient encounters: a validation study of the Interpreter Scale. Acad Med. 2009;84(5):643–650. https://doi.org/10.1097/ACM.0b013e31819faec8Google Scholar
McEvoy M, Santos MT, Marzan M, Marzan M, Green EH, Milan FB. Teaching medical students how to use interpreters: a three year experience. Med Educ Online. 2009;14:12. https://doi.org/10.3402/meo.v14i.4507Google Scholar
Omoruyi EA, Dunkle J, Dendy C, McHugh E, Barratt MS. Cross talk: evaluation of a curriculum to teach medical students how to use telephone interpreter services. Acad Pediatr. 2018;18(2):214–219. https://doi.org/10.1016/j.acap.2017.11.010Google Scholar
Vargas Pelaez AF, Ramirez SI, Valdes Sanchez C, et al.Implementing a medical student interpreter training program as a strategy to developing humanism. BMC Med Educ. 2018;18(1):141. https://doi.org/10.1186/s12909-018-1254-7Google Scholar
Trial J, Elliott D, Lauzon V, Lie D, Chvira E. Interpretation at the OB/GYN bedside—cultural competence in the third-year clerkships. MedEdPORTAL. 2010;6:1148. https://doi.org/10.15766/mep_2374-8265.1148Google Scholar
Zabar S, Hanley K, Kachur E, et al.“Oh She doesn't speak English” Assessing resident competence in managing linguistic and cultural barriers. J Gen Intern Med. 2006;21(5):510–513. https://doi.org/10.1111/j.1525-1497.2006.00439.xGoogle Scholar
Waldman S, Kalet A. Working with interpreters: learning to conduct a cross-language medical interview with an online web-based module (out of print). MedEdPORTAL. 2007;3:654. https://doi.org/10.15766/mep_2374-8265.654Google Scholar
Farris S. Preeclampsia in a post-partum woman. Council of Residency Directors in Emergency Medicine website. https://www.cordem.org/globalassets/files/sim-oral-board-cases/13.0obgyn/f_preeclampsia_farris_final.docx. Accessed August 30, 2019.Google Scholar
- 24. The Emergency Medicine Milestone Project: a joint initiative of the Accreditation Council for Graduate Medical Education and the American Board of Emergency Medicine. Accreditation Council for Graduate Medical Education website. https://www.acgme.org/Portals/0/PDFs/Milestones/EmergencyMedicineMilestones.pdf. Published July 2015. Accessed September 3, 2018.Google Scholar
Sullivan GM. A primer on the validity of assessment instruments. J Grad Med Educ. 2011;3(2):119–120. https://doi.org/10.4300/JGME-D-11-00075.1Google Scholar
Sibai BM. Diagnosis, prevention, and management of eclampsia. Obstet Gynecol. 2005;105(2):402–410. https://doi.org/10.1097/01.AOG.0000152351.13671.99Google Scholar