Original Publication
Open Access

Engaging and Educating Resident Physicians in Patient Safety and Peer Review

Published: May 20, 2016 | 10.15766/mep_2374-8265.10401

Appendices

  • Faculty Training Presentation.pptx
  • Resident Small-Group Presentation.pptx
  • Case Assignment Email.docx
  • Guide to Mentoring Peer Review.docx
  • Survey Instrument.pdf

All appendices are peer reviewed as integral parts of the Original Publication.

To view all publication components, extract (i.e., unzip) them from the downloaded .zip file.


Abstract

Introduction: This patient safety curriculum was developed to address two important needs in contemporary graduate medical education: training frontline providers in key patient safety skills and producing tangible products that demonstrate resident achievement in this area. The curriculum is geared towards second- and third-year internal medicine residents. Methods: It employs a mixed-methods approach to teaching patient safety concepts by integrating standardized introductory cases with individualized real-time cases under consideration by the department’s peer review committee. This resource consists of an instructional slide set and facilitator guide for the patient safety and peer review course, including standardized cases of medical error. Also included are a practical workflow depicting faculty and resident training, as well as guidance on the necessary collaboration with local peer review leadership. Results: Following the training of 14 faculty members and curriculum completion by 60 residents, preliminary resident before-after survey results demonstrate a significant increase in the percentage of residents who report being able to complete a mortality review, feeling comfortable with key components of the curriculum (e.g., teaching the difference between active and latent failures, recognizing the difference between levels of standard of care), and having a high level of comfort with the concept of peer review (either performing it on a colleague’s case or having a colleague perform it on one of their own cases). Discussion: Integrating residents into ongoing departmental patient safety efforts can provide residency programs with a novel way to teach essential patient safety concepts as well as to meet current competency requirements.


Educational Objectives

After completing this course, residents will be able to:

  1. Identify the difference between active and latent failures.
  2. Describe the value of systems thinking in the setting of potential patient harm.
  3. Distinguish the different levels of standard of care.
  4. Assess an adverse event using the peer review framework.
  5. Prepare a written peer review.
  6. Reflect upon the role of peer review in clinical practice.
  7. Recognize the role of frontline providers in improving patient safety.

Introduction

Despite being frontline providers in complex systems and frequent witnesses of failures of those systems, resident physicians have historically been absent from patient safety work. This may stem from a lack of exposure to tools for evaluation of adverse events and patient safety mentorship. In the setting of evolving accreditation requirements, many residency programs may also be looking for ways to engage residents in patient safety work in a quantifiable way.1

In 2013, we performed a needs assessment of our internal medicine residency quality and patient safety curriculum. We had an existing monthly quality conference focused on reviewing medical errors and discussing objective outcome and operational metrics, a resident quality committee for those interested in obtaining quality improvement mentoring, and the opportunity to apply for quality-related research grant funding through the department of medicine. At that time, program leadership sought to balance the interest of residents in obtaining additional hands-on experiences in the area of patient safety with the administrative need to provide clear documentation of these experiences. At the same time, departmental quality leadership was also considering how to best involve frontline providers in the peer review process. This curriculum was subsequently developed to address these educational, administrative, and patient safety needs.

Key learning points and curricular materials were designed by departmental administrative and educational leaders and partly derived from their practical and administrative expertise. Additionally, a literature review was undertaken to ensure the learning points and educational approach were consistent with current practices in the area of patient safety education.2-8 There have been several excellent MedEdPORTAL publications that have provided a key framework and inspiration for our project. Cumbler and Glasheen introduced a robust curriculum that used medical errors as the basis for teaching key patient safety topics.3 Forstater, Levinson, Bellot, Hess, and Spandorfer utilized a single hypothetical case to teach team communication skills and an approach to systems thinking via root cause analysis.6 Goolsarran then encouraged incorporating the analysis of real-time cases into standardized small-group discussion sessions as a method for teaching patient safety.8

Our approach aimed to combine key concepts from these curricula. We utilized a standardized medical error case to introduce patient safety topics and then followed it with a mentored review of a real-time case under consideration by the department’s peer review committee. To further engage residents, we integrated their work into the existing patient safety infrastructure by inviting them to present their cases to the department’s peer review committee and participate in discussions with department safety leadership. This allowed learners to observe how the patient safety principles they learned via standardized cases and their real-time review were regularly applied on an administrative level by department leadership.

We initially ran a limited pilot of the course. Feedback was elicited from residents and faculty mentors with respect to course design, comprehension of key learning points, and perceived effectiveness of course formatting.

We now present an internal medicine resident curriculum on patient safety and peer review that was implemented at the Icahn School of Medicine at Mount Sinai in 2013 and has subsequently been held on a monthly-to-bimonthly basis thereafter. As we felt that a strong clinical background and experience as a frontline provider would enhance a learner’s ability to comprehend key patient safety concepts and relate them to everyday care, we identified our goal audience as PGY-2 and PGY-3 internal medicine residents. The curriculum has been designed to address two important needs in contemporary graduate medical education. The first is engaging frontline resident providers in the peer review process as part of building and applying their patient safety skill set. The second is producing tangible products that demonstrate resident comprehension and application of key related concepts.

Methods

Given that teaching patient safety and peer review frequently inspires discussion of sensitive topics (e.g., medical errors), we opted for a discussion-focused small-group approach for the initial encounter, followed by an experiential component in the form of a peer review. As the independent peer cases showed significant variation in clinical and patient safety content, we felt that subsequent patient-specific discussions would have the highest yield if completed on a one-on-one basis. Additionally, we believed this would be most consistent with our goals of creating a close mentorship between faculty member and resident. Lastly, as we hoped to engage residents in the existing departmental patient safety and quality improvement infrastructure, we concluded our curriculum with an invitation to participate in the department’s peer review committee process.

This resource details a small-group activity, hands-on independent peer review, mentored one-on-one case assessment, and formal peer review committee case presentation (see the Figure).

Figure. Patient safety and peer review course
workflow.

The course enrolled PGY-2 and PGY-3 internal medicine residents who were scheduled for a 4-week block of an elective clinical rotation. Residents attended a lunchtime small-group case-based didactic on adverse events and peer review, during which time key patient safety topics were introduced. The 60-minute session was led by a course director, who was knowledgeable about patient safety concepts being taught as well as the practical aspects of the institution’s peer review process. The session also utilized audiovisual equipment to present the resident small-group PowerPoint (Appendix B), as well as a whiteboard or flip chart during periods of discussion. Audience size for this session was three to six residents and was run on a monthly basis. Key objectives for this case-based section were to identify the difference between active and latent failures, discuss the types of systems failures and contributing factors, and distinguish the different levels of standard of care. The session culminated with performing a mock peer review, including identification of relevant medical details, possible systems failures, and contributing factors, as well as making a determination regarding standard of care.

Following completion of the didactic session, each resident was assigned a faculty mentor and a real-time case under consideration by the departmental peer review committee. A templated form of the e-mail containing details of the assignment is included in Appendix C. Residents were subsequently responsible for reviewing and producing a draft of a formal write-up of the assigned case using the peer review guidelines discussed during their small-group didactic. Residents and faulty mentors were then asked to schedule a one-on-one session.

In preparation for this mentoring session, all potential faculty mentors initially underwent a training session. Contents of this training included material similar to that included in the resident presentation, supplemented with relevant information and discussion about teaching patient safety. The 60-minute session was held during a regularly scheduled faculty development noon conference. It was led by a course director and utilized audiovisual equipment to present the faculty training PowerPoint (Appendix A) and a whiteboard or flip chart during periods of discussion. Goal audience size was eight to 12 faculty members. Subsequent introductory sessions and refresher sessions were held as needed based on faculty hiring and turnover (one to two sessions per year, 10 faculty in attendance on average). All participating faculty were provided with a facilitation guide (Appendix D) consisting of key expectations and learning points. The facilitation guide was digitally redistributed to faculty via e-mail at regular intervals.

Subsequently, each resident-faculty pair met to discuss the case, review the resident’s written product, and reinforce key concepts. Sessions were scheduled for 30 minutes and required both the faculty and resident to have reviewed and prepared for discussion regarding key case details and learning points as well as determination of standard of care. Sessions required a quiet room, but otherwise, no additional equipment was necessary.

Revised written products were submitted by residents via the department’s secure online mortality review tool. All submissions were reviewed by department peer review leadership. Resident and faculty mentors were invited to present their case to the department peer review committee and (if applicable) engage in discussions regarding potential systems solutions. All reviews were stored securely by the peer review committee and made available to the residency program leadership for inclusion as part of a digital patient safety portfolio.

Before and after this curriculum, participating residents were sent optional and anonymous SurveyMonkey surveys via e-mail (Appendix E). With input from residency program leadership, patient safety leadership, and involved medical educators, the surveys were designed to assess patient safety attitudes and self-reported comprehension of learning objectives. A focus group of residents had assessed survey questions for reliability of resident understanding.

Results

As of the time of submission, 60 PGY-2 and PGY-3 internal medicine residents have completed the peer review and patient safety course and had their write-ups made available as part of a digital patient safety portfolio. Additionally, 14 faculty members have completed faculty training and have acted as mentors.

Of the 60 participating residents, 50 responded to the precourse survey (83.3%), and 19 responded to the postcourse survey (31.7%; see the Table). In a preliminary chi-square analysis of precourse responses compared to postcourse responses, significant increases were observed in the proportion of respondents agreeing or strongly agreeing with the following statements:

  • I am capable of performing a mortality review.
  • I could teach a colleague about the difference between active and latent failures.
  • I can recognize the difference between meeting standard of care, meeting standard of care with room for improvement, and failing to meet standard of care.
  • I feel comfortable reviewing mortality cases that my coresidents may have cared for.
  • I feel comfortable with my coresidents reviewing mortality cases that occur on my service.
Table. Pre- and Postcourse Survey Responses 
Item
Precourse
n (%)
Postcourse n (%)
pa
Responses
50 (83.3)
19 (31.7)
Demographics
 PGY-2
25 (50.0)
5 (26.3)
 PGY-3
25 (50.0)
14 (73.6)
Agree or Strongly Agree n (%)
Agree or Strongly Agree n (%)
Question
 Mortality review is a valuable skill for all physicians to learn.
49 (98.0)
17 (89.5)
.12
 I am capable of performing a mortality review.
17 (34.0)
19 (100)
<.001
 When a patient dies unexpectedly, I have a tendency to think about mistakes the     providers may have made.
46 (92.0)
19 (100)
.10
 When a patient dies unexpectedly, I have a tendency to think about systems errors     that may have happened (team factors, work environment, organizational factors).
47 (94.0)
19 (100)
.13
 I could teach a colleague about the difference between active and latent failures.
17 (34.0)
19 (100)
<.001
 I can recognize the difference between meeting standard of care, meeting standard
    of care with room for improvement, and failing to meet standard of care.
30 (60.0)
19 (100)
<.001
 I feel comfortable reviewing mortality cases that my coresidents may have cared for.
28 (56.0)
19 (100)
<.001
 I feel comfortable with my coresidents reviewing mortality cases that occur on my     service.
40 (80.0)
19 (100)
.02
aUnadjusted p value calculated via chi-square.

In response to written anonymous feedback, the curriculum was modified to make all course materials available to resident participants prior to the small-group session and to optimize the timing of the initial small-group discussion (the earlier in the clinical schedule block, the more time the residents would have to review their case and meet with their mentor).

Discussion

Following the completion of our patient safety and peer review course, a preliminary survey analysis showed a significant change in reported patient safety attitudes and self-assessed comprehension of course learning objectives amongst participating internal medicine residents.

We found that engaging faculty and residents in patient safety work also yielded visible cultural change in the valuing of patient safety and peer review. Once faculty mentors became involved in teaching patient safety concepts, they frequently inquired about becoming more involved in the administrative side of the peer review process. In fact, one such faculty member now helps lead the departmental peer review committee. Similarly, residents who completed this curriculum frequently looked for additional ways to be involved with the peer review process. We received multiple requests to develop a system for communicating the results of peer review to all involved providers, even when no preventable harm was identified. Determining how to longitudinally involve residents in peer review, as well as the systems-redesign process, is the next goal for our curriculum.

We initially only targeted learners in our outcomes measures. Given the amount of focus we have also placed on training and engaging faculty in the area of patient safety education, we are now defining faculty-focused outcomes so that we can monitor and improve faculty training, maximize mentor effectiveness, and better understand how residents respond to feedback during mentorship sessions. Current results do not reflect these planned future innovations.

Presently our residency peer review curriculum relies on the reviewing of objective data and documentation. Residents are given the option, but are not required, to speak with involved providers (the primary team, consultants, nurses) in order to better understand aspects of the case that may not appear in the written record. Through feedback, residents have shown a strong interest in actually contacting those providers involved in cases. We aim to standardize and provide additional training regarding how to interact with and provide support to providers who may have been involved in a medical error. In the next revision of our survey instrument, we also plan to include a question about whether they chose to discuss their case with involved providers and how this was received.

Recently, our medical center has become part of a larger health system. As efforts are made to standardize administrative processes across the system (potentially including peer review), we hope to find ways to spread our approach to the other residency programs within our new network.

Resident interest seemed to be partially driven by the inclusion of real-time cases for peer review. Implementing this portion of the curriculum required close coordination between the department’s peer review leadership, the residency program’s Associate Program Director for Quality, and the patient safety medical educators. In our case, such a curriculum was seen as furthering the interests of each party involved. Peer review leadership saw this as an important component of improving our culture of safety. Residency leadership saw this as a hands-on learning experience congruent with evolving accreditation requirements. Educators saw this as an opportunity to augment existing patient safety and quality improvement learning. In settings where the interests and goals of these parties are not aligned, implementation of such a curriculum may prove challenging. Even in our well-aligned environment, we have struggled to match up residency and administrative scheduling to ensure that residents are available to present at the peer review committee. So far in our group of 60 residents, only 16 who have been invited were able to attend. This remains an area for improvement.

Despite the positive responses we received from residents, we encountered difficulty obtaining responses to our anonymous and optional survey. As response rate may pose a barrier to ensuring the continued improvement of our patient safety curriculum, we are exploring other methods of eliciting regular feedback. Data collection and analysis, including more in-depth before-after survey data comparisons, remain ongoing.


Author Information

  • Joshua Allen-Dicker, MD, MPH: Instructor-in-Medicine, Department of Medicine, Harvard Medical School
  • Brian Markoff, MD: Associate Professor, Department of Medicine, Icahn School of Medicine at Mount Sinai
  • Vinh-Tung Nguyen, MD: Assistant Professor, Department of Medicine, Icahn School of Medicine at Mount Sinai
  • Brian Radbill, MD: Associate Professor, Department of Medicine, Icahn School of Medicine at Mount Sinai
  • Brijen Shah, MD: Assistant Professor, Department of Medicine, Icahn School of Medicine at Mount Sinai; Assistant Professor, Department of Geriatrics and Palliative Medicine

Disclosures
None to report.

Funding/Support
None to report.

Ethical Approval
Reported as not applicable.


References

  1. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366(11):1051-1056. http://dx.doi.org/10.1056/NEJMsr1200117
  2. Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298(9):1023-1037. http://dx.doi.org/10.1001/jama.298.9.1023
  3. Cumbler E, Glasheen J. Teaching patient safety via a structured review of medical errors: a novel approach to educating residents about medical error, disclosure, and malpractice. MedEdPORTAL Publications. 2007;3:761. http://dx.doi.org/10.15766/mep_2374-8265.761
  4. Tess AV, Yang JJ, Smith CC, Fawcett CM, Bates CK, Reynolds EE. Combining clinical microsystems and an experiential quality improvement curriculum to improve residency education in internal medicine. Acad Med. 2009;84(3):326-334. http://dx.doi.org/10.1097/ACM.0b013e31819731bf
  5. Wong BM, Etchells EE, Kuper A, Levinson W, Shojania KG. Teaching quality improvement and patient safety to trainees: a systematic review. Acad Med. 2010;85(9):1425-1439. http://dx.doi.org/10.1097/ACM.0b013e3181e2d0c6
  6. Forstater A, Levinson M, Bellot J, Hess M, Spandorfer J. Patient safety symposium: issues, analyses, prevention. MedEdPORTAL Publications. 2013;9:9637. http://dx.doi.org/10.15766/mep_2374-8265.9637
  7. Tad-y D, Price L, Cumbler E, Levin D, Wald H, Glasheen J. An experiential quality improvement curriculum for the inpatient setting—part 1: design phase of a QI project. MedEdPORTAL Publications. 2014;10:9841. http://dx.doi.org/10.15766/mep_2374-8265.9841
  8. Goolsarran N. Patient safety education curriculum for medicine residents. MedEdPORTAL Publications. 2015;11:10208. http://dx.doi.org/10.15766/mep_2374-8265.10208


Citation

Allen-Dicker J, Markoff B, Nguyen V-T, Radbill B, Shah B. Engaging and educating resident physicians in patient safety and peer review. MedEdPORTAL. 2016;12:10401. https://doi.org/10.15766/mep_2374-8265.10401

Received: December 23, 2015

Accepted: April 7, 216