A Novel Simulation to Assess Residents' Utilization of a Medical Interpreter

Educational Objectives

By the end of this activity, faculty will be able to:

Introduction

Non-English-speaking patients face significant barriers to care and have a lower satisfaction with their physician-patient relationship.^1–8 However, the use of a qualified medical interpreter (MI) improves patient and provider satisfaction and is mandated for communication when the provider is not fluent in the patient's preferred language.^9–14 As a result, physicians must be facile in working with an MI in the clinical setting.

We performed a literature review of MEDLINE (using PubMed, keywords medical interpreter simulation and medical interpreter resident) and MedEdPORTAL (keywords medical interpreter), which yielded several curricula focused on the use of MIs. Many of the prior curricula were geared toward medical student education, and although some incorporated standardized patients, none utilized simulation to assess learner competence in the use of interpreters.^15–20 We found two curricula specifically designed for residents, but neither included a significant assessment component.^21,22

The Accreditation Council for Graduate Medical Education's (ACGME's) Clinical Learning Environment Review process mandates education on reducing health care disparities. Despite this mandate and the prior published curricula, a survey of residents in pediatrics residency programs found that 54% never received formal education in the use of MIs. Additionally, there is a dearth of tools for assessing resident competence in this skill. Thus, we developed a low-stakes, formative simulation assessment tool to address health care of the non-English-speaking patient.

To evaluate and improve emergency medicine residents' skills in working with an MI, we developed a simulation scenario to allow for multisource assessment of their ability to communicate with a Spanish-speaking family member in an emergency situation. We chose the use of simulation over other modalities (e.g., direct assessment of patient encounters) for the ability to standardize the encounter for all residents and for the ability to obtain a multisource standardized assessment of the residents' performance. To implement this scenario, we adapted a case we had previously used that involved a patient with eclamptic seizures and made a certified MI available to the residents during the case.²³

Methods

Development

Recognizing the need to assess our residents' ability to use MIs according to ACGME Emergency Medicine Milestone 22, Patient Centered Communication,²⁴ we adapted a previously existing case of eclamptic seizures and incorporated the need for an MI by making the patient and her family non–native English speaking. The case was developed by the program directors and departmental simulation director in conjunction with the program's education committee. Assistance was provided by the university's simulation staff and the MIs who ultimately participated in the case.

This case was implemented as part of our emergency medicine residency's semiannual formative simulation assessment program. As part of this program, each resident participated in two simulation assessment cases every 6 months. All cases were calibrated to be challenging for a senior resident to complete successfully. Results were used in conjunction with other evaluative data to identify residents in need of additional support or remediation and to assess resident competency. Validity evidence for the assessment tool was compiled before delivering the data to the clinical competency committee for use in resident milestone-level assignments.²⁵ The assessment tool and simulation case were reviewed by multiple board-certified emergency medicine faculty prior to pilot testing, with adjustments made to incorporate feedback, lending content validity.

Equipment/Environment

Equipment used in the simulation included the following:

Personnel

Personnel in the simulation included the following:

Implementation

The case took place in a high-fidelity simulation laboratory. Briefly, the case involved a Spanish-speaking female of childbearing age who had given birth the prior week and then developed eclamptic seizures. The full case script is provided in Appendix A. Each case took approximately 15 minutes, and each learner completed the case once. Approximately 20 cases were completed per room per day.

Setup included briefing the MI, the simulated participant who played the patient's Spanish-speaking husband or sister (see Appendix B), the nurse simulated participant, and the faculty observer. Upon arrival at the simulation area but prior to entry into the room, the learner was briefed on standard rules of our institution's simulation laboratory. The learner was then provided with the case stem (see Appendix A) before entering the simulation room. No additional review material was given to the learner prior to the evaluation.

The MIs were recruited from the local children's hospital interpreter pool, as well as from a local volunteer group. All were certified Spanish MIs. The interpreter department director was contacted, and the voluntary opportunity was offered to all certified Spanish MIs. The MIs were oriented to the case on the morning of the assessment. Further preassessment training of MIs was determined to be unnecessary, as they were asked to participate in the case in their typical professional role. Prior to the formal start day of the assessments, the MIs reviewed the script to familiarize themselves with the flow of the simulation, and they participated in the final walk-through to ensure their familiarity with the case. The simulated family member and nurse simulated participant were recruited from our medical school simulation center's pool of standardized actors. The family member was chosen for his bilingual fluency, realistic portrayals in other simulation activities, and experience. Specific attention was placed on his demeanor, as it was important for him to maintain his professionalism and remain in character despite potentially negative interactions during the simulation. The nurse simulated participant was chosen for her critical care background, experience with emergency medicine simulation, and ability to provide realistic portrayals as an emergency medicine nurse.

Both simulated participants were given copies of the case to be memorized before training began. Training consisted of a tabletop walk-through with both simulated participants, faculty leaders, and simulation operations staff. Following this, the case was run several times in the simulation room with faculty portraying the residents. These sessions took 2–3 hours and occurred prior to the first day of formal assessments. A final walk-through of the case was completed on the morning of the first day of assessments as a final check to ensure all participants were familiar and comfortable with their roles.

Faculty observers were recruited from within the department. The 15 faculty who participated in rating this case were provided with the case and assessment tool, as well as general information on the structure of the assessment, prior to the testing day. Upon arrival, they were again briefed on the case by the faculty leads, and any questions were answered. Faculty observers were not involved in running the case; they performed only ratings and debriefing.

Initially, only the nurse simulated participant and simulated patient were in the room with the learner; the simulated family member and MI were out of sight in the control room. The learner received the Emergency Medical Services report verbally from the simulation operator but was not made privy to the patient's recent obstetric history. The patient was post-ictal and unable to provide a history in any language, only moaning during examination of her abdomen. Shortly after the case began, the patient seized again, prompting the learner to administer an antiepileptic. During the seizure, the patient's Spanish-speaking husband arrived in the room, was distraught, and immediately sought information from the learner on his wife's status. As the husband demonstrated concern and requested information in Spanish, the learner had to determine how to communicate effectively with him while continuing to medically manage the seizing patient.

If the learner requested interpretive services, the MI arrived in the room. The learner could then use the MI to obtain history from the patient's husband that strongly suggested eclampsia as the cause for her seizures. Laboratory, radiology, and other diagnostic results are detailed in Appendix C. Ideally, the learner should then have ordered magnesium and treated the patient's elevated blood pressure. Otherwise, she would continue to seize and would develop status epilepticus despite the administration of other antiepileptic drugs. The gynecologist, if called, would initially try to avoid immediate consultation by stating he or she was in the OR. However, if the learner was insistent, the specialist would agree to respond immediately. The case ended after the gynecologist accepted and admitted the patient to the ICU, magnesium was given, hypertension was appropriately treated, and the patient's husband was updated with her diagnosis and plan. Otherwise, the case ended after 15 minutes.

Assessment

All residents were evaluated on observable actions that were anchored to the language of the Emergency Medicine Milestones.²⁴ The learner was assessed by both the faculty observer and the MI who participated in the case. The MI's assessment (Appendix D) focused on the resident's interpersonal communication skills, including whether the resident requested an MI or was already fluent in Spanish, utilized the MI properly by maintaining eye contact with and speaking directly to the family, and used lay terminology in a culturally sensitive manner. Although we did not utilize the simulated family member for feedback, the MI assessment form could easily be adapted for use by this person.

The faculty observer's assessment (Appendix D) focused on the broader patient care, including whether the resident performed a primary resuscitation assessment, completed all important elements of the history and physical examination, ordered appropriate testing, appropriately recognized and treated the eclamptic seizure, and consulted the gynecologist on call. The faculty observer also evaluated the resident's communication skills. Critical actions were created based on the ACGME Emergency Medicine Milestones.²⁴

Debriefing

After the case, the learner was debriefed in a separate classroom by the faculty observer. The observer reviewed the case and the learner's actions and clarified any ambiguous actions that had occurred during the simulation. The observer also reviewed the critical actions and provided verbal feedback on the learner's performance. The correct diagnosis and treatment were discussed, and the learner was given the opportunity to explain his or her thought process during the case. Finally, the appropriate use of an MI was discussed, including the need to use short phrases, pause frequently for interpretation, and speak directly to the patient (or family member).

Probing questions during the debriefing included the following:

Appendix E contains a tip sheet for the use of an MI clinically. Although we did not utilize this appendix with our learners, we created it as a resource for educators to use when implementing this case. Additionally, the cited work serves as a good overview of the diagnosis and treatment of eclampsia for both faculty reference and learner review.²⁶

Results

The simulation assessment was performed with 60 emergency medicine residents, supervised by emergency medicine faculty, in fall 2016. Only four residents did not request an MI, two of whom were native speakers. Both native speakers were evaluated for their ability to communicate clearly with the patient and family and were given an evaluation of “unable to assess” for their use of the MI. Common areas for improvement were identified regarding the use of lay terminology, as well as maintaining proper eye contact and body positioning, when using an MI. Compared to the prior version of the case that utilized the same eclamptic seizure scenario but did not include the language barrier element, residents received lower overall scores on the new version. Overall scores on the modified case were a median of 10 out of 27 (odds ratio = 0.43; 95% confidence interval, 0.18–0.99) compared to a median score of 12 out of 24 achieved on the prior version, with a significant difference between the percentage of total items correct for the two groups (p = .02). This suggests that the inclusion of a language barrier created a more challenging case for the residents.

Effectiveness Data

Median scores on the assessment were noted to decrease by PGY subgroup (PGY 3: 20.5; PGY 2: 19.5; PGY 1: 15.0), showing a split between junior and senior residents and lending evidence for construct validity of this simulation as an assessment tool. Response process validity was addressed through extensive simulation operator and standardized patient training, as well as instructing raters (interpreter and faculty) on the proper use of the assessment tool. Scores for the subcompetency Interpersonal and Communication Skills 1, which addresses the use of interpreters in communication, were similar between the interpreter and the faculty raters (median score of 5 overall and per each class, maximum possible score of 6).

Discussion

To conduct a formative assessment of emergency medicine residents on the use of MIs, we adapted a previously used case of eclamptic seizures to include a non-English-speaking patient and relative. Overall, residents scored lower on this version of the assessment than on the prior version that omitted the language barrier, and areas of improvement for the learners included adopting the standard mechanics of using an interpreter.

The process of adding an MI to a previously developed case is challenging but would be possible if an instructor wishes to introduce a language barrier to a different simulation scenario. On reflection, we believe that a moderately difficult case in which key elements of the history must be obtained via an interpreter would allow for maximal emphasis on the importance of proper interpreter use. We also added an element of emotional distress when the Spanish-speaking family member entered the room in order to challenge the learner to manage emotions while obtaining a history with the MI. This element could be omitted in a different case if the instructor wishes to limit the added challenge to the language barrier alone. In any case in which a language barrier is used, it is important to allocate sufficient time for the learner to recognize the need for interpretive services, for the MI to appear in the room, and for the history to be obtained using a third person. Finally, a case could be similarly modified to use an interpreter over the phone. In such a case, the trained MI could be positioned in the control room, with a double-handset phone, tablet computer, or other similar device used for communication.

There were several challenges that we did not anticipate during implementation of the case. During the debriefings, it became clear that some residents had not expected an MI to be available, with some not requesting an MI and later stating they otherwise would have. As a result, we suggest that the door prompt or nurse script include a cue that MI services are available upon request or that all ancillary services typically available at the hospital are available.

Additionally, several of our Spanish-speaking residents appropriately chose to conduct the interview without the MI. If an instructor wishes to prevent this from occurring in the future, the instructor could modify the case to utilize another language that is not known to any of the learners. This, of course, would require an assessment of the languages in which learners are fluent. It could also lead to additional difficulty in finding interpreters and patient actors who are comfortable working in the chosen language.

Limitations

There are several limitations to this assessment as it currently exists. First, we designed the assessment for formative purposes, with an emphasis on learner education. As such, we did not control for variability between MI and faculty observations. Additionally, to facilitate participation by MIs, their orientation to the case and assessment system was minimal and could have resulted in limited interrater reliability. Although we believe that a similar case could be used for high-stakes assessment, several preimplementation changes would be required. These include further validation of the scoring system and the creation of a standardized training mechanism for evaluators. Furthermore, we designed this case specifically for emergency medicine residents, and the underlying diagnosis may be too difficult or esoteric for residents in unrelated specialties.

Implications

Following the implementation of this case, we found that many of our residents did not know the basic mechanics of using an MI. This anecdotal finding correlates with the literature finding that fewer than half of residents have received formal training in the use of MIs, and it represents an area of opportunity to improve the care of vulnerable patient populations.

Although this case was created specifically for the training of emergency medicine residents, it could be adapted for use by other specialties, such as obstetrics and family medicine, as well as for use in the training of medical students. Other specialties could utilize a similarly designed case with a disease process specific to their area of expertise. We believe that the pathology underlying the case is less important because the primary focus of the case is on the appropriate use of an MI.

The use of MIs is an integral part of health care practice in the United States. We present a simulation case that can assess learners' use of MIs and allow for an educational intervention that may improve the care of non-English-speaking patients.