Original Publication
Open Access

A Teaching OSCE to Assess Trauma Resident Skills on How to Hold Difficult Conversations With Family of Critically Injured Patients

Published: May 18, 2016 | 10.15766/mep_2374-8265.10398

Appendices

  • TOSCE Trauma Case 1.docx
  • TOSCE Trauma Case 2.docx
  • TOSCE Outline.pptx
  • Learner Assessment Form.doc
  • Pre-Post Evaluation.doc

All appendices are peer reviewed as integral parts of the Original Publication.

To view all publication components, extract (i.e., unzip) them from the downloaded .zip file.


Abstract

Introduction: Traumatic injury often results in death or significant disability to a previously healthy person, leaving family overwhelmed. Effective communication to support family is therefore important, yet there is little emphasis placed on developing communication skills around death and dying in the trauma setting. Assessment of a resident’s communication skills or feedback regarding these skills is rare to nonexistent in emergency medicine and surgery resident training. We designed a curriculum to teach and assess communication skills that address difficult conversations in the trauma bay, which includes this teaching objective structured clinical examination (TOSCE) used to assess communication skills. Methods: After a brief introduction and orientation to peer feedback, residents were divided into groups of three to complete two 7-minute TOSCEs, where one resident interacts with a standardized patient (SP) and the other two observe. Communication and overall demeanor are assessed using a checklist format and on a Likert-type scale. The same form was used by the learner, the SP, and the facilitator. SPs assessed residents, and peers/preceptors provided formative feedback. Results: Twenty-five residents (nine emergency medicine, 16 surgery) participated in the TOSCE. A majority of participating residents were rated by the standardized patients as competent to perform independently, while a small percentage needed more basic instruction. For the two case scenarios used, SPs rated nearly all residents as competent to perform delivery of poor prognosis and competent or mostly competent to perform delivery of news of death. Mean ratings showed concordance between self-ratings and SP ratings. A majority of residents reported the TOSCE was a valuable learning experience. Discussion: In conclusion, a TOSCE is effective in assessing communication skills around the critically injured patient in the trauma bay. A TOSCE is feasible to implement in surgery and emergency medicine residency training, and the format is valued by participants.


Educational Objectives

At the end of this session, the learner will be able to:

  1. Demonstrate the basic approach to exploring and handling emotions in families, including active listening, empathic statements, use of silence, and appropriate nonverbal behaviors.
  2. Demonstrate the delivery of difficult news of a patient’s death in the trauma setting.
  3. Demonstrate the delivery of difficult news of a patient’s poor and/or uncertain prognosis to family in the trauma setting.
  4. Understand use of self-reflection to identify own strengths and weaknesses regarding communication skills.

Introduction

Traumatic injury is an out-of-time and out-of-place event that affects people of all ages. This sudden event is likely to affect a previously healthy and young person, thus leaving surviving family unprepared and in disbelief.1-5 The trauma setting has unique challenges when it comes to delivery of difficult news. The clinician (emergency medicine [EM] or surgery) who delivers this difficult news of death and/or disability has no prior relationship with the family.2,4,5 The clinician has to communicate in a chaotic environment, with time constraints, and often in the face of a rapidly evolving clinical situation. It is not surprising that the delivery of difficult news in trauma settings is accompanied by intense emotions in both the clinician, who may have just completed an intense resuscitation, and the family, who may be angry, in denial, or in shock.5 Therefore, effective communication is crucial in the trauma bay to help support family and survivors.

Training of residents in trauma focuses mainly on the technical and procedural skills of advanced trauma life support (ATLS), so teaching of communication skills may not be a priority.1-5 Direct observation of resident performance or feedback to a resident on his or her communication technique is rarely provided in EM and surgery training.1-5 Structured assessment of resident communication skills, such as with use of an objective structured clinical examination (OSCE), is rare to nonexistent in both EM and surgery residency training.2 An educational needs assessment for EM trainees has highlighted the fact that EM residents identify end-of-life discussions and managing the imminently dying patient as major topics of interest.1 Based on a comprehensive literature review, we found limited studies that describe outcomes related to assessment of resident communication skills. We identified a specific gap in both teaching and assessing of resident communication skills in the trauma setting.2

With generous funding from a graduate medical education grant by the Gold Picker Foundation, we designed and implemented a three-phase curriculum to address this gap. Phase one is a didactic session with use of e-learning.5 Phase two is the practice of both of the technical trauma resuscitation skills closely linked with relevant communication skills using hybrid simulation. Phase three is assessment and feedback with a teaching OSCE (TOSCE; for an outline, see Appendix C), including self-assessment.

In this resource, we describe the TOSCE assessment of resident communication skills in the trauma setting. The TOSCE session was designed to ensure that each trainee performed the procedure of a difficult conversation with a standardized patient (SP), self-assessed his or her own performance, and then received assessment and feedback from the SP. The trainees also received formative feedback from their peers and the faculty facilitator (the teaching part of the TOSCE) along with a copy of a video of their performance.

This TOSCE was designed for junior to midlevel EM and surgery residents. It can be easily adapted for use with other trainees in the trauma setting, such as medical students, physician assistants, and nursing learners. Our pilot was designed for EM and surgery PGY 1 and PGY 2 residents (we included PGY 1-PGY 4 EM residents in the pilot).

Some basic knowledge of resuscitation in trauma, such as familiarity with basic ATLS skills and care for the patient, may be beneficial, as would an understanding of basic concepts regarding doctor-patient communication. Ideally, this TOSCE assessment resource should be preceded by a didactic session reviewing basic effective communication strategies and the unique communication needs in the trauma setting, followed by an opportunity to practice these skills (especially if this OSCE is used for summative assessment). Prior to the TOSCE assessment, we delivered a didactic session with the use of e-learning5 and offered a hybrid simulated session where an ATLS-based resuscitation of a simulated patient was followed by role play to practice delivering difficult news to the simulated patient’s family. The TOSCE scenarios were designed to also be able to function stand-alone.

Methods

Time limitations have created a need to explore settings other than the bedside, such as simulated encounters, to fulfill or document acquisition of core competencies that are traditionally more challenging to measure such as communication, lifelong learning, and professionalism.4-8 Our overall project goals were to teach and to assess the skills of effective communication during the unique high-stress, high-stakes conversations that follow trauma resuscitations. We used a TOSCE to directly observe and assess the communication skills and to provide targeted formative feedback to learners. We grounded our curriculum in Kolb’s experiential learning theory. The TOSCE, or assessment phase of our curriculum, closely mirrors the last two phases of stage four of Kolb’s spiral of learning, which ensures that the learner touches all the bases of thinking, experiencing, acting, and reflecting.9 The e-learning module and the practice of skills in a simulated setting portion5 mainly activate prior knowledge and assist in applying knowledge with practice of skills, while the TOSCE mainly focuses on the acting and reflecting phases.

Case 1 (Appendix A) covers delivery of difficult news of death to the family (played by an actor/SP) of a trauma victim. Case 2 (Appendix B) covers delivery of difficult news of poor prognosis to the family of a trauma victim. Page one of both case files contains information for the facilitator including purpose of the case, training level (resident), simulated patient name, diagnosis, setting, and time allotted. Page two contains background information for residents including setting and patient clinical scenario, along with the task description and time allotted. Page three contains the SP (family member) script.

The TOSCE was held in our clinical skills center. The residency programs included the session as part of the mandatory didactics on Wednesday morning (a day dedicated to didactics for both resident programs). Twenty-five residents participated in the pilot. We found that three learners per group worked well since one resident performed the encounter and received feedback from two peers and a faculty facilitator. This resident in turn observed the other two encounters. This did mean that there was a repeat of the first scenario, but since our purpose was formative feedback, we instructed the SP to make the second similar encounter more challenging and emotional with the use of denial and anger. Please note that repeating a case has some inherent limitations, especially if the OSCE is used for summative assessments. One suggested workaround may be to have one resident sit out the first encounter.

There was a brief 10-minute introduction in the common room to orient residents to the objectives of the session, discuss flow and logistics, and address questions and concerns. It was emphasized that each resident should treat the SP as a real family member of the patient and examine and interact with the SP as the resident normally would in a real setting. The residents were also notified that patient interaction would start as soon as they entered the room, there would be a time prompt at 5 minutes into the session, they would receive written feedback from the SP during debrief, and they would be expected to observe the encounter of a peer and provide thoughtful feedback.

We spent time defining what was expected when the residents provided peer feedback, such as use of a nonjudgmental tone and a positive approach to discussion. We used the Guidelines on How to Provide Feedback, included in a previous MedEdPORTAL publication by Lamba, Offin, and Nagurka,4 as an outline for  residents and faculty. In particular, we discussed being descriptive rather than judgmental, being specific rather than using vague terms, being sensitive to needs of peers, and directing feedback to a behavior rather than the person.

Residents were then divided into groups of three with mixed training levels and mixed disciplines. Each OSCE session was a 7-minute encounter. Each resident read the introductory narrative on the room door and walked in to perform the communication scenario while the others observed the encounter. There was a warning bell at 5 minutes.

Communication and overall demeanor are assessed using a checklist format (Appendix D) and on a Likert-type scale. The same form was used by the learner, the SP, and the facilitator. Appropriate checkboxes on the form identify the case scenario assessed and whether an evaluator is learner, peer, or faculty.

The pre-post questionnaire (Appendix E) may be used to assess baseline learner comfort and confidence with skills, as well as for a brief overall evaluation of the TOSCE session. Please note that in our project, the residents completed the prequestionnaire prior to delivery of an entire curriculum. The postsurvey was done after three stages of curriculum: the didactics, the simulation with role play, and the TOSCE. Here, we present a modified version of the evaluation that could be used with a stand-alone TOSCE.

Preparation for TOSCE
The two residency programs met to determine an appropriate time, book a clinical skills center, and discuss logistical issues. We conducted the TOSCE on Wednesday morning, the usual set didactic time for surgery and EM residents. Of note, all the EM residents were included since there were no specific separate targeted activities for senior residents scheduled, whereas surgery did have separate didactics for senior residents.

The faculty facilitators and clinical skills director codeveloped the SP scripts and recruited SPs. The final SP script was edited based on feedback from the SPs and a mock encounter. For example, our initial script had two SPs per encounter and allowed for 15 minutes per encounter. The mock encounter led to adjusting the time for each case, and we decided to use one SP per encounter since the use of two did not add any value to the interaction (but added to cost). The group session was held with SPs to train them on both cases in detail. We discussed goals of the session and familiarized SPs with the assessment checklist as well as feedback format. For example, they were asked to place less emphasis on the technical conversations but focus more on the emotions of loss of a loved one when in SP mode. The mock encounter was discussed for feedback role modeling.

Next, the blueprints for the day were created and reviewed. Printed forms were made available for SP, for faculty, and on a clipboard for each resident. We had a dedicated audiovisual person for recording, as well as issuing time prompts for the beginning and ending of the encounter and the end of debriefing. We also had two clinical skills personnel to help orient and facilitate the flow of residents. To minimize loss of clinical time and idle time for residents, they were scheduled in batches per hour. We had two faculty facilitators for communicating feedback, a surgeon and an EM physician. Both had expertise in palliative care and communication skills training. Facilitators met for a mock session with an SP and a resident volunteer to determine timing and flow and discuss a structured way to provide feedback for both cases.

We then discussed a common way to debrief. For example, one technique is for the SP to use the checklist to provide directed feedback. The facilitator then asks the resident what went well and what did not go well in the encounter. This is followed by asking what the resident would do differently for a similar real-family encounter. The peers next provide feedback based on their observations, and finally, a faculty facilitator adds summary comments. Other debriefing methods such as ask-tell-ask or the ECO (emotions, content, outcomes) approach may be suitable for this part of the exercise.

Results

Twenty-five residents participated in the TOSCE, nine EM residents (PGY 1-PGY 4) and 16 surgery residents (PGYs 1 and 2). A majority of residents (92%) reported that the TOSCE was a valuable learning experience.

Overall, SPs rated 78% of residents (18 out of 23) as competent to perform independently, 9% (two out of 23) as needing close supervision, and 13% (three out of 23) as needing basic instruction. Note that two residents were missing overall performance ratings by the SPs.

By scenario, SPs rated 92% of residents (11 out of 12) as overall competent to perform independently giving bad news in the poor prognosis scenario (one resident was rated as needing basic instruction). SPs rated 64% (seven out of 11) as overall competent to perform independently giving bad news in a death scenario, while two residents were rated as needing close supervision and two residents were rated as needing basic instruction.

Range of mean ratings across domains was 2.15-2.92 on the Likert-type scale of 0 to 3 (0 = unsatisfactory or poorly done, 3 = excellent). For the death scenario, lowest mean rating (2.25) was for “resident presented in a professional manner” and “resident expressed empathy.” For the poor prognosis scenario, the lowest mean rating (2.15) was for “resident expressed understanding.”

Mean ratings showed concordance between the self-rating by resident and those by the SP. It is interesting to note that there was a slight but not significant trend for higher rating by SPs in all domains except “professional manner” and “expressed empathy” in the death scenario and “professional manner” and “expressed understanding” in the poor prognosis scenario. For these domains, the residents rated themselves higher than the SP rating.

We also performed a pre-post assessment; the results are presented in the Table. Please note that the preassessment was done before the curriculum began. The curriculum included didactics, simulation and role play, and TOSCE. Postassessment was therefore performed after the entire curriculum was completed.

Table. Pre-Post Learner Responses for Curriculum 
Response
Precurriculum (N = 39)
Postcurriculum (N = 28)
p < .05
Mdn
M (SD)
Mdn
M (SD)
Comfort with breaking bad news
 Death
3.00
3.08 (0.84)
3.00
3.21 (0.57)
ns
 Prognosis
3.00
3.03 (0.78)
3.00
3.25 (0.59)
ns
Confident with breaking bad news
 Death
3.00
3.15 (0.78)
3.00
3.32 (0.72)
ns
 Poor prognosis
3.00
3.15 (0.74)
3.00
3.29 (0.66)
ns
Knowledge with breaking bad news
 Death
3.00
3.08 (0.81)
4.00
3.50 (0.58)
.0453
 Poor prognosis
3.00
3.03 (0.74)
3.00
3.43 (0.57)
.0439
Knowledge with teaching about . . . bad news
 Death
2.00
2.38 (1.04)
3.00
2.50 (0.79)
ns
 Poor prognosis
2.00
2.31 (0.89)
3.00
2.46 (0.84)
ns
Abbreviation: ns, nonsignificant.

Discussion

The OSCE format is commonly used in undergraduate medical education, but its use to assess the communication skills of surgery and EM residents is rare. To our knowledge, this is the first description of an OSCE for assessment and direct observation of communication skills for EM and surgery residents in the unique trauma setting. The TOSCE was a novel format for our residency programs. Not only was it feasible but a majority of the residents took the experience very seriously and had multiple positive comments such as “The TOSCE was a valuable exercise” and “This should happen early and more often in residency training.” Even the senior residents expressed that “this was the first time they had ever received feedback on their communication skills” and found value in the exercise. Interestingly, the only skepticism that teaching and assessing communication skills would be of interest to surgery and EM residents actually came anecdotally from some faculty members. This skepticism regarding value of communication skills among select faculty may need to be managed in programs that recruit different faculty to debrief the TOSCE or teach content related to communication skills. In summary, as residency programs look for ways to assess competency and milestone achievements, particularly for difficult-to-measure items such as communication skills, professionalism, and the ability to give and receive feedback as lifelong learners, this TOSCE would be a useful resource.

Lessons learned include the suitability of this case scenario for junior to midlevel residents, our target learners. When used for senior residents, we recommend training the SPs to add strong emotions (denial, shock, and anger worked well in our TOSCE). The layering of more complex emotions was valuable, made the encounter more challenging to navigate, and mirrored reality in the trauma environment where distraught family members often react with anger and occasionally with violence. Though the residents liked the staggered schedule, since this allowed them to arrive for the TOSCE with no downtime for waiting and to leave as soon as it was done, we had to repeat orientation as each batch of residents arrived for their session.

Use of two faculty (surgery and EM) with expertise in palliative care and communication skills training and both well-known in the clinical setting to the surgery and EM residents made the feedback process smooth. If more faculty participate, particularly those who are not familiar to residents, we recommend a more detailed process of faculty development before facilitation.

Some residents felt that when we practiced a simulated patient’s trauma resuscitation closely linked with role play of delivering the news, it felt much more realistic and relevant as compared to the TOSCE, where we used the traditional written narrative on a handout that described the patient’s clinical course. As we now develop an advanced communication skills assessment, we plan to address this resident concern. We plan to have the resident team (three to four members) perform a brief ATLS-based simulated trauma resuscitation and then proceed to deliver the news to family (SPs) in TOSCE. Though this creates an added step, we feel closely linking the performance of the technical aspects and the assessing of communication may be more real and better suited to the resident-level learners.

The SPs we use are very familiar and interact routinely with medical student–level trainees. Medical students are asked to wear business attire and white coats for OSCEs. We feel the residents may have been inappropriately rated lower by the SPs on their professional manner due to their attire. Some residents were going off-call, and some were in scrubs or in jeans (due to it being a didactic day). SPs commented on this during the debriefing after the TOSCE, and therefore, we plan to address this bias and better prepare our SPs for resident interactions.

We also felt on debriefing with the SPs after session that SPs who are very familiar with medical student learners may have perceived resident confidence as being a bit arrogant (faculty preceptors thought the resident confidence was training appropriate). We plan to address this disconnect with better SP training.

Logistically, coordinating a TOSCE date with surgery and EM residency programs and SP scheduling was a bit challenging. We found that cycling the surgery residents earlier (7:00 a.m. to 8:30 a.m.) and then the EM residents worked best for clinical duties. We recommend setting a date and schedule at least 2 months in advance due to the logistical pieces.

The staggered scheduling worked well for us since all residents were punctual and we did not have any idle time or filler content to consider.


Author Information

  • Sangeeta Lamba, MD: Emergency Medicine Attending Physician, Rutgers New Jersey Medical School
  • Sarah Bryczkowski, MD: Surgery Resident, Rutgers New Jersey Medical School
  • Bart Holland, PhD: Biostatistics Faculty, Rutgers New Jersey Medical School
  • Roxanne Nagurka: Research Assistant, Rutgers New Jersey Medical School
  • Anne Mosenthal, MD: Surgery Attending Physician, Rutgers New Jersey Medical School

Disclosures
None to report.

Funding/Support
None to report.

Ethical Approval
This publication contains data obtained from human subjects and received ethical approval.


References

  1. Lamba S, Pound A, Rella JG, Compton S. Emergency medicine resident education in palliative care: a needs assessment. J Palliat Med. 2012;15(5):516-520. http://dx.doi.org/10.1089/jpm.2011.0457
  2. Lamba S, Tyrie LS, Bryczkowski S, Nagurka R. Teaching surgery residents the skills to communicate difficult news to patient and family members: a literature review. J Palliat Med. 2015;19(1):101-107. http://dx.doi.org/10.1089/jpm.2015.0292
  3. Mosenthal AC, Murphy PA. Trauma care and palliative care: time to integrate the two? J Am Coll Surg. 2003;197(3):509-516. http://dx.doi.org/10.1016/S1072-7515(03)00651-3
  4. Lamba S, Offin M, Nagurka R. Case-based simulation: crucial conversations around resuscitation of the critically-ill or injured patient. MedEdPORTAL Publications. 2013;9:9367. http://dx.doi.org/10.15766/mep_2374-8265.9367
  5. Tyrie L, Mosenthal A, Bryczkowski S, Laboy C, Lamba S. Difficult conversations after resuscitation in trauma: video education e-module. MedEdPORTAL Publications. 2015;11:10092. http://dx.doi.org/10.15766/mep_2374-8265.10092
  6. Bowyer MW, Hanson JL, Pimentel EA, et al. Teaching breaking bad news using mixed reality simulation. J Surg Res. 2010;159(1):462-467. http://dx.doi.org/10.1016/j.jss.2009.04.032
  7. Larkin AC, Cahan MA, Whalen G, et al. Human Emotion and Response in Surgery (HEARS): a simulation-based curriculum for communication skills, systems-based practice, and professionalism in surgical residency training. J Am Coll Surg. 2010;211(2):285-292. http://dx.doi.org/10.1016/j.jamcollsurg.2010.04.004
  8. Gettman MT, Karnes RJ, Arnold JJ, et al. Urology resident training with an unexpected patient death scenario: experiential learning with high fidelity simulation. J Urol. 2008;180(1):283-288. http://dx.doi.org/10.1016/j.juro.2008.03.042
  9. Kolb DA. Experiential Learning: Experience as the Source of Learning and Development. Upper Saddle River, NJ: Prentice Hall; 1983.


Citation

Lamba S, Bryczkowski S, Holland B, Nagurka R, Mosenthal A. A teaching OSCE to assess trauma resident skills on how to hold difficult conversations with family of critically injured patients. MedEdPORTAL. 2016;12:10398. https://doi.org/10.15766/mep_2374-8265.10398

Received: January 5, 2016

Accepted: April 5, 2016