Introduction: We created a standardized workshop to engage residents in quality improvement (QI) using the root cause analysis model. The workshop allows for a robust learning experience while providing solutions derived from clinicians to address important local problems. No prerequisite knowledge or experience is required. Methods: The workshop is facilitated by one or more moderators, ideally with experience in QI. An interdisciplinary group of residents, medical students, nurses, and other attendees comprise an audience which actively engages in workshop activities. Facilitators follow a scripted model to teach important patient safety concepts with frequent break-outs for hands-on application of QI tools. During the workshop, participants create a process map and fishbone diagram, as well as develop and critically evaluate novel interventions. Results: Over the course of one academic year, the workshop has been implemented 17 times with roughly 25 internal medicine residents in attendance at each workshop. In addition, the workshop was run online for 126 participants with varied exposure to QI techniques. Forty percent of these participants completed a survey indicating that over 89% learned something new, 87% felt they could apply the material to their work, and 95% would recommend the workshop to a colleague. Discussion: This 60-minute workshop can provide hands-on QI experience in a standardized format to achieve the dual objectives of teaching QI to clinicians and allowing them to generate innovations. The module can be used for internal case development and trainee participation, but prepared cases are provided for facilitators without the resources for local case development.
- Define and compare the Agency for Healthcare Research and Quality definitions for adverse events, medical errors, and near misses, and apply these definitions to a local adverse event.
- Understand and construct a fishbone diagram based on a local adverse events.
- Apply the concepts of the hierarchy of interventions to a case-based patient safety report.
- Categorize solutions to a case-based patient safety issue using an impact/effort matrix, prioritizing interventions based on relative efficacy and feasibility.
- Engage in collaborative and multidisciplinary problem solving based on a local adverse event.
Meaningful system improvement in healthcare is most achievable when informed by the experience of frontline clinicians such as medical students, residents, attending physicians, nurses, and others immersed in direct patient care. Traditional root cause analyses (RCAs) include representatives from the care teams involved with the case to uncover the root cause of systemic failure. Anita L. Tucker noted that frontline employees provided valuable input on selecting problem “resolution efforts” as a result of having “in-depth knowledge” about processes and equipment failures.1 With the high volume of incidents filed that require investigation, formal RCAs are often foregone. As a consequence, the lack of a frontline perspective can result in missed opportunities for improvement. Collective and systematic involvement by frontline employees addressing smaller problems not visible to management results in value-added efforts concerning effective systems improvement.1 The 60-minute RCA is a novel attempt to fill this need by involving frontline clinicians in systems improvement on a regular basis through resident engagement and education.
In 2011, the ACGME recognized the importance of both patient safety and creating a safe environment for patients, concluding that current education in quality and patient safety is inadequate.2 The organization developed the Clinical Learning Environment Review (CLER) focusing on six aspects of resident education, including patient safety and quality improvement (QI), to address the noted deficiencies. In particular, a major finding from CLER was that “across [clinical learning environments], a limited number of residents, fellows, and faculty members participated in inter-professional, interdisciplinary, systems-based improvement efforts such as patient safety event reviews and analyses.”3-5
While CLER has established the requirement of QI and patient safety education as part of residency education, there has been no consensus on how best to provide training in QI and patient safety. The activities that satisfy the requirement are varied and nonstandard. Effective teaching of QI includes a mixed approach using didactics, demonstrations, and workshops to effectively impart quality QI tools.6 Internal medicine residents have increased exposure to QI through didactic teaching and online modules, such as those from the Institute for Healthcare Improvement.7,8 However, it has been more difficult for trainees to apply this knowledge and create new skills on a regular basis.
The authors have created this standardized workshop to engage internal medicine residents in the investigation and implementation of improvement work using the RCA. The RCA is a well-studied and frequently used QI tool in the systematic assessment of an adverse event.9 The workshop allows for a robust learning experience while providing frontline-derived solutions to important local problems.10
There is no prerequisite knowledge necessary to complete the workshop. The process includes education on a number of key concepts needed to evaluate real-time patient-safety issues, coupled with key skills that are practiced in small group breakout sessions with support from the moderator and facilitators.
Specific key concepts taught by the moderator include:
- Define and compare the Agency for Healthcare Research and Quality (AHRQ) definitions for adverse events, medical errors, and near misses, and apply these definitions to a local adverse event.
- Understand the fishbone/Ishikawa diagram.
- Apply the concepts of the hierarchy of interventions to case-based patient safety report.
- Understand and interpret an impact/effort matrix.
Specific key skills taught by the moderator include:
- Complete a fishbone diagram based on a local adverse event.
- Categorize solutions to a case-based patient safety issue using an impact/effort matrix.
- Engage in collaborative and multidisciplinary problem solving based on a local adverse event.
Since 2006, a 3-week rotation in patient safety in the Veterans Affairs Boston Healthcare System has been a component of the internal medicine residency curriculum at Boston Medical Center. Through June 2016, this rotation culminated in a 1-hour patient safety noon conference, led by the rotating resident and focusing on work they performed during the rotation. These sessions were heterogeneous in nature and often didactic in delivery. To address concerns that the conference was not educational for audience members, key stakeholders met to develop and implement a revamped workshop.
This newly designed 60-minute RCA workshop used educators in multiple unique roles (Table). The role of moderator was filled by the rotating resident in patient safety. The role of facilitator was filled by the local Veterans Affairs chief resident in quality and patient safety or a faculty member trained in QI.5 The role of the focus group was filled by rotating internal medicine residents, rotating medical students from Boston University School of Medicine and Harvard Medical School, and other attendees with interdisciplinary representation, often including nursing and pharmacy. The total size of the focus group ranged between 20-40 clinicians.
Person Filling Role
Rotating PGY 2 or 3 resident
Present the case and guide the workshop following a standardized structure.
Chief medical resident or faculty
Provide quality improvement expertise and insight.
Internal medicine residents (PGY 1-3), medical students, and other interdisciplinary attendees, including pharmacy, nursing, and others as appropriate
Participate in breakout sessions to practice quality improvement skills and produce solutions to local systems problems.
Prior to the workshop, the moderator was tasked with researching a case from the incident reporting system and performing a literature review pertaining to relevant patient safety issues raised by the case. Details on how to obtain, investigate, and prepare a case for presentation are provided in Appendix E. A case can be selected based on clinical necessity, organizational preference, or clinician preference. Once selected, documenting the event through process mapping and a detailed timeline of facts was required as part of necessary prework. Depending on the case in question and the available resources, this investigation varied in depth from hospital to hospital. The moderator outlined the case objectively and allowed the focus group participants to highlight system errors.
During the workshop, the moderator began by briefly summarizing the case from a systems perspective, utilizing tools such as a process map (a step-by-step process of the event) to reflect the in-depth case investigation and inform the focus group of the circumstances surrounding the event. The moderator specifically taught the AHRQ definitions of adverse events, medical errors, and near misses (Key Concept 1), and applied these definitions to the event in question. The moderator actively directed the conversation in the room away from any in-depth clinical questions and decision points, and instead maintained focus on systems vulnerabilities. This introduction and review of the case took no more than 15 minutes.
The moderator then taught the concept of a fishbone diagram and assisted the workshop members in building their own arms of the fishbone. The facilitator and moderator cotaught this concept (Key Concept 2, details in Appendix A) without identifying specific systems errors related to this case. The fishbone diagram evaluates the patient safety issue from the perspective of multiple categories. Each category represents an arm of the fishbone pertaining to the event. Examples of categories include human factors, environment, policy and procedures, education and training, to name a few. This was then followed by a breakout into smaller focus groups to brainstorm systems barriers under specific categories on the fishbone diagram (Key Skill 1). Finally, members reported out to the group at large, creating a unique fishbone diagram for this case. These steps (Key Concept 2 and Key Skill 1) took no more than 20 minutes. Of note, Appendix A provides detail on how to approach and teach the fishbone diagram with the goal of enabling even those with minimal QI exposure to lead the workshop, with slide 11 containing a visual depiction of the diagram.
Following this discussion, the moderator and facilitator cotaught the concept of high vs. low-impact interventions (Key Concept 3). The focus groups reconvened to brainstorm interventions, applying the high-impact teaching. The moderator/facilitator then taught the concept of an impact-effort matrix (Key Concept 4). The focus groups reported their recommendations, their responses were scored on the matrix (Key Skill 2), and this discussion provided a rich understanding of systems perspectives in medicine. During this portion of the workshop, the members of the focus group engaged in collaborative problem solving with wide representation from medicine, nursing, pharmacy, and other services as appropriate for the case (Key Skill 3). Teaching Key Concepts 3 and 4 and Key Skills 2 and 3 used the remainder of the hour, taking approximately 20 minutes.
Materials: Necessary materials included whiteboard/chalkboard, pens, paper, and printouts of fishbone/Ishikawa diagrams. Included in this submission are four PowerPoint files with case examples, including entire case investigations and teaching points for QI topics. Appendix A provides the entire structure for a case review. The speaker’s notes section of the PowerPoint includes bolded text to provide instruction and move the workshop forward. The PowerPoint presentations should be used in presenter mode so the presenter can read the text and utilize the built-in animations; reviewing the files in the editing mode will miss a great deal of the content. Appendices B through D are presentation-ready cases from our local medical center. Workshops using these cases resulted in strong engagement from learners.
Provided cases: The cases include a delayed transfer to the cardiac care unit resulting in harm (Appendix B), a delayed transfusion that did not result in overt harm (Appendix C), and a case of a supratherapeutic heparin drip that resulted in harm (Appendix D). We have included Appendices B through D so the reader can readily present this workshop with a widely applicable case. However, one purpose of this workshop is to produce frontline innovation relevant to local institutions; and it would be ideal to obtain local cases and adapt them to this format using Appendix A as a framework. We suggest seeking out patient safety reports filed by residents as a source for these cases. Details on obtaining, investigating, and preparing a case for presentation can be found in Appendix E.
Attendees: Personnel invited to the workshop include internal medical residents (years one through three), medical students, and representation from nursing, pharmacy, and other healthcare providers and personnel as available and appropriate to each investigation. Management and leadership should be included in the sessions when possible.
Learners were assessed as a group through the workshop itself based on the quality of the fishbone and impact/effort diagrams. Included in the appendices are examples of completed assessments. Given the nature of the workshop, these group assessments were preferred as compared to typical individual questionnaires targeting discrete facts.
Between June 2016 and May 2017, this 60-minute RCA workshop has been implemented 17 times with roughly 25 internal medicine residents in attendance at each workshop. Three core faculty members were present at most workshops and contributed as needed, two with expertise in QI and one with a focus on graduate medical education.
This workshop has a dual purpose. The first is to teach QI skills and leverage the frontline expertise of residents and other care providers to create meaningful improvement in the local healthcare system. The results of the workshops can best be described by the quality of the fishbone and impact/effort matrices that are created de novo by the participating residents at each workshop. Representative examples of the completed fishbone diagrams are provided in Appendices B, C, and D and the impact/effort matrix is in Appendix D.
The second purpose, meaningful improvement, has been abundantly evident at our institution. This workshop has resulted in numerous changes to practice including streamlined resident hand-offs and critical care unit transfers, the implementation and spreading of nurse-driven heparin protocols, improved time to antibiotics for sepsis, the designing of an opiate withdrawal protocol, and improved communication with the blood bank. Many of the QI projects that impact residents at our institution have been initiated through these workshops.
In November 2017 the workshop was run as a webinar for 126 participants interested in QI. These participants did not include medical students or residents, but rather individuals across a spectrum of disciplines not limited to physicians. The participants had a varied experience with QI and included six administrators, three full-time clinicians, seven clinical educators, and 15 QI professionals. A survey accompanying this webinar was completed by 40% of the participants (all four questions were answered by either 48 or 49 participants out of 126 attendees).
Results from the survey demonstrated that 89.8% of participants learned something new from the workshop, while 87.5% felt they would be able to apply the knowledge to their own work. Likewise, 95.9% of participants would recommend the session to a colleague, and 85.4% felt the content was appropriate for their needs. For further details please refer to Figure 1.
The experience with use of the QI tools used in the workshop was heterogenous. Only 23.5%, 21.7%, and 31.1% of participants were very experienced with the fishbone diagram, hierarchy of interventions, and impact/effort matrix, respectively. The majority of participants had only some, little or no experience with all three tools. Results are demonstrated in Figure 2.
This workshop was designed as an extensive overhaul of an existing patient safety conference. The original conference was designed for residents to showcase QI work performed during their QI/safety rotation. Unfortunately, it often became too focused on didactics or a description of work that had already been completed and did not feel sufficiently relevant and engaging to the audience. The general sentiment and informal feedback suggested that there was minimal QI/safety learning.
With the general impetus to move towards interactive learning,11 we redesigned the conference as a workshop to allow learners to simulate elements of an RCA investigation and actively engage them in the process of improvement utilizing newly learned skills. The results demonstrate that across an audience with variable QI backgrounds, the material is relevant, accessible, and appropriate. The vast majority of those surveyed learned new knowledge that can be applied to their practice. Additionally, the authors noted significantly increased attendance and vibrant engagement at the workshop. Residents that rarely attended or participated in conferences were suddenly vocal contributors. Choosing cases that specifically involved the residents was a key component of this process. Resident buy-in was natural during cases that focused on systemic barriers or frustrations that they encountered in their daily work.
Using the provided case examples would make for a successful workshop at any institution. However, applying the model to local cases, ideally from problems reported by residents or other frontline staff, is likely to result in more successful engagement and teaching. Beyond the educational value, creating a forum where frontline clinicians could apply themselves to the systems problems they face on a daily basis resulted in more robust and innovative solutions that could be trialed by the patient safety and QI teams.
The biggest challenge we faced was time. This workshop could easily run for 90 minutes with great success, but we were limited to a 60-minute schedule. It is possible to run the workshop in 60 minutes and we have done this with robust discussion and innovation for all 17 of the cases presented in 2016-2017. However, we have experimented with alternatives to get more out of the workshop, such as teaching the related medical content in the morning report on the same day or providing optional pre- and postwork.
The conference has adapted significantly over time. It became more streamlined in the presentation of the case and data, increasingly moving from the sharing of medical details to a focusing on systems details. An emphasis has been placed on presenting local data, such as surveys of local residents, nurses, or technicians, and on easily retrievable data from chart reviews or data mining. We found that the time was best spent when used in work groups and guided discussion.
Our next step is to utilize asynchronous feedback tools such as Poll Everywhere or other survey tools to obtain feedback after the conferences.
None to report.
None to report.
Reported as not applicable.
- Tucker AL. An empirical study of system improvement by frontline employees in hospital units. Manuf Serv Operations Manag. 2007;9(4):492-505. https://doi.org/10.1287/msom.1060.0156
- Accreditation Council for Graduate Medical Education. Focus on the future: Accreditation Council for Graduate Medical Education 2011 annual report. https://www.acgme.org/Portals/0/PDFs/ACGME-2011_AR_F.pdf. Published 2012.
- Weiss KB, Bagian JP, Wagner R. CLER pathways to excellence: expectations for an optimal clinical learning environment (executive summary). J Grad Med Educ. 2014;6(3):610-611.
- Myers JS, Nash DB. Graduate medical education’s new focus on resident engagement in quality and safety: will it transform the culture of teaching hospitals? Acad Med. 2014;89(10):1328-1330. https://doi.org/10.1097/ACM.0000000000000435
- Watts BV, Paull DE, Williams LC, Neily J, Hemphill RR, Brannen JL. Department of Veterans Affairs Chief Resident in Quality and Patient Safety Program: a model to spread change. Am J Med Qual. 2016;31(6):598-600. https://doi.org/10.1177/1062860616643403
- Aron DC, Headrick LA. Educating physicians prepared to improve care and safety is no accident: it requires a systematic approach. Qual Saf Health Care. 2002;11(2):168-173. https://doi.org/10.1136/qhc.11.2.168
- Miller R, Winterton T, Hoffman WW. Building a whole new mind: an interprofessional experience in patient safety and quality improvement education using the IHI Open School. S D Med. 2014;67(1):17-19, 21-22.
- Miller RJ, Hoffman WW. “Heart Bone”—the case for the IHI Open School as connector: a model for integrating quality improvement and patient safety into health professions curricula. S D Med. 2015;68(6):245-250.
- Bagian JP, Gosbee J, Lee CZ, Williams L, McKnight SD, Mannos DM. The Veterans Affairs Root Cause Analysis System in action. Jt Comm J Qual Improv. 2002;28(10):531-545. https://doi.org/10.1016/S1070-3241(02)28057-8
- Palis AG, Quiros PA. Adult learning principles and presentation pearls. Middle East Afr J Ophthalmol. 2014;21(2):114-122. https://doi.org/10.4103/0974-9233.129748
- Armstrong G, Headrick L, Madigosky W, Ogrinc G. Designing education to improve care. Jt Comm J Qual Patient Saf. 2012;38(1):5-14. https://doi.org/10.1016/S1553-7250(12)38002-1
This is an open-access publication distributed under the terms of the Creative Commons Attribution-NonCommercial-Share Alike license.
Received: September 22, 2017
Accepted: January 29, 2018