Original Publication
Open Access

Cognitive Debiasing Strategies: A Faculty Development Workshop for Clinical Teachers in Emergency Medicine

Published: October 23, 2017 | 10.15766/mep_2374-8265.10646

Appendices

  • Cognitive Debiasing Strategies for Teachers.pptx
  • Facilitator Guides for Cognitive Debiasing Cases.docx
  • Cognitive Debiasing Cases.docx
  • Pocket Card.pdf
  • Cognitive Bias Codex.docx
  • Workshop Evaluation.docx

All appendices are peer reviewed as integral parts of the Original Publication.

To view all publication components, extract (i.e., unzip) them from the downloaded .zip file.


Abstract

Introduction: Medical decision-making is a cornerstone of clinical care and a key contributor to diagnostic accuracy. Medical decision-making occurs via two primary pathways: System 1, pattern recognition, is fast, intuitive, and heuristically driven and occurs largely unconsciously. System 2, analytic thinking, is slow, deliberate, and under conscious control. Biases are systematic errors that can impact reasoning via either pathway but predominantly affect decisions made by pattern recognition. Debiasing strategies involve the deliberate switching from pattern recognition to analytic thinking triggered by a stimulus. This resource describes a faculty development workshop designed to train emergency medicine educators about common biases and debiasing strategies, to improve teaching of diagnostic reasoning to trainees. Methods: This workshop was implemented at the 2017 Society for Academic Emergency Medicine Annual Meeting. The workshop consisted of a brief didactic, followed by small-group case-based learning. A retrospective survey and qualitative evaluation were administered to attendees. Results: The participants’ self-assessment showed significant improvements (p < .001) in their abilities to recognize how pattern recognition can lead to bias, identify common types of bias in the emergency department, teach trainees about common types of bias, and apply cognitive debiasing strategies to improve diagnostic reasoning. Strengths of the workshop included the interactive case-based format, discussions of bias-mitigation strategies, and take-home resources. Suggestions for improvement included lengthening the discussion time and providing more cases. Discussion: Cognitive biases can negatively impact patient care. Faculty development is needed to improve instruction about bias and debiasing strategies for all levels of trainees.


Educational Objectives

By the end of this activity, learners will be able to:

  1. Recognize how the use of illness scripts, heuristics, and pattern recognition can lead to bias.
  2. Identify common types of bias in the emergency department.
  3. Outline strategies to teach trainees about common types of bias.
  4. Apply cognitive forcing or debiasing strategies to improve diagnostic reasoning.

Figures and Tables

Table 1. Results of Retrospective Evaluation (4-point scale; 1 = Not at All, 2 = A Little, 3 = Moderately, 4 = Very)
Skill
Before
After
Difference
p
Effect size (r)
Please rate your ability for the following skills:
 Recognize how pattern recognition can lead to bias.
2.74
3.67
0.93
<.001
.57
Identify common types of bias in the emergency department.
2.56
3.56
1.00
<.001
.57
 Teach trainees about common types of bias.
1.93
3.04
1.11
<.001
.59
Apply cognitive forcing or “debiasing” strategies to improve diagnostic reasoning.
2.22
3.41
1.19
<.001
.62

Introduction

The emergency department is a high-acuity environment where a need for speed combined with conditions of uncertainty can lead to unacceptably high diagnostic failure rates. Across specialties, diagnostic error rates are on the order of 10%-15%,1 though there are likely specialty-specific differences. An estimated 70% of diagnostic errors are due to faulty reasoning.2 An extensive body of research has examined the causes of faulty reasoning at the individual clinician level. This work has been informed by research in the field of cognitive psychology, which examines how doctors process information.

According to dual process theory, clinical decision-making occurs through two primary information-processing pathways: Pattern recognition (System 1) is fast, intuitive, and heuristically driven and occurs largely unconsciously. Analytic or logical thinking (System 2) is slow and deliberate and takes place under conscious control.3-5 When functioning optimally, physicians toggle back and forth between these two systems6 depending on the complexity of the case and certain ambient conditions (e.g., time pressures, multitasking, cognitive overload, change of shift, fatigue, sleep deprivation, etc.).

Biases are systematic errors that can interfere with diagnostic reasoning. Over 180 cognitive and affective biases have been described in the lay press and scholarly journals,7,8 many of which are common in emergency medicine.9 Some experts believe that pattern recognition is more prone to bias, but empiric evidence suggests that bias can be associated with both processing systems.10 Since expert clinicians operate about 95% of the time in pattern-recognition mode, most error-reduction or debiasing strategies focus on System 1 thinking.

The concept of cognitive debiasing has been described in detail by Croskerry, Singhal, and Mamede.11,12 Debiasing strategies are designed to force practitioners out of pattern recognition into a more analytic mode of thinking, providing a mental correction to optimize decision making. Several debiasing strategies have been described.13 When trained about bias and debiasing strategies, providers may be better able to recognize high-risk stimuli to switch from System 1 to System 2 thinking and have the ability to employ error-mitigation strategies.

Prior MedEdPORTAL publications on diagnostic error, cognitive bias, and debiasing strategies have focused on teaching learners to identify errors, while equipping them with tools to combat bias. Stiegler and Goldhaber-Fiebert described a video intervention coupled with a debriefing session to help learners identify common cognitive biases.14 They focused on helping learners normalize the experience of being wrong, identifying strategies for prevention and earlier recovery from cognitive error. Chew, van Merrienboer, and Durning utilized case-based scenarios to facilitate identification of common biases and taught final-year medical students to use a reflection mnemonic as a debiasing strategy.15 Ruedinger, Mathews, and Olson introduced residents to diagnostic error and cognitive strategies to reduce error through personal narratives and reflective practice.16

Efforts to train learners about cognitive bias and debiasing strategies should be augmented through faculty training to help ensure transfer of concepts into clinical practice. The currently available resources focus largely on medical student and resident education. More faculty development resources are needed. This resource complements prior MedEdPORTAL offerings by addressing this faculty development gap. The workshop uses case-based active-learning strategies to help emergency medicine clinician educators improve their understanding of cognitive bias and debiasing strategies with the express aim of augmenting their ability to teach these concepts to a wide range of learners.

Methods

This 60-minute workshop was implemented at the Society for Academic Emergency Medicine (SAEM) Annual Meeting in 2017. The workshop consisted of an interactive didactic, case-based small-group discussions, and a facilitated report-out.

A single meeting room was reserved with multiple tables that could accommodate six to eight people each. The room was equipped with a projector, a lavalier, and a portable microphone. A lead presenter was identified to deliver the opening presentation (Appendix A). This presenter rehearsed the presentation several times to ensure a fluid delivery, using the presenter notes section of the PowerPoint for guidance. Faculty facilitators were recruited (one per table). Facilitators read selected literature on cognitive debiasing (specifically, an article by Daniel, Khandelwal, Santen, Malone, and Croskerry9 and two by Croskerry, Singhal, and Mamede12,13), reviewed the facilitator guide for cognitive debiasing cases (Appendix B), and prepared to discuss the biases and mitigation strategies highlighted by each case. The average preparation time was 3 hours, depending on prior education and training in cognitive bias and debiasing strategies.

Packets were assembled for each attendee consisting of the case files (Appendix C), the pocket card (laminated and sized to fit in a white-coat pocket; Appendix D), the Cognitive Bias Codex (Appendix E), and the workshop evaluation (Appendix F). The packets were distributed to each attendee upon arrival.

The lead presenter delivered the PowerPoint presentation (Appendix A), which took about 15 minutes. This brief interactive didactic reviewed dual process theory, cognitive biases, and debiasing strategies. Embedded in the presentation were a large-group activity and think-pair-share exercise (Slides 8-14) that highlighted one type of bias and strategies to mitigate it. Participants were then divided into small groups for a facilitated discussion (30 minutes). They applied the concepts learned to cases highlighting different cognitive biases, including but not limited to triage cueing, diagnosis momentum, confirmation bias, premature closure, anchoring, search satisficing, psych-out error, and availability bias.

After reading through a case, participants discussed two key questions: “What types of cognitive bias are at play at various stages in this case?” and “How could this bias be mitigated proactively if you could do it over?” Participants were encouraged to refer to the pocket card and Cognitive Bias Codex during the discussion. Small groups completed two cases—either Cases 1 and 2 or Cases 3 and 4—as these pairings provided the most diverse array of common biases. The session closed with a facilitated report-out from each small group of the most significant take-home points (15 minutes).

A retrospective survey was distributed to participants at the conclusion of the session to assess learning. Participants were asked to rate their abilities before and after the session on a 4-point scale (1 = Not at All, 2 = A Little, 3 = Moderately, 4 = Very). The rationale for using a retrospective survey rather than a traditional pre/post methodology for self-assessment was summarized by Bhanji, Gottesman, de Grave, Steinert, and Winer.17 The items were developed by an expert in evaluation methodology and then reviewed and revised by the author group for content and response process validity evidence. Differences in the means were calculated, as were effect sizes. Cronbach’s alpha was assessed on the posttest as a measure of internal consistency. All statistics were performed in Microsoft Excel. In addition to the survey questions, participants were asked to qualitatively describe what contributed to their learning in the workshop and what could be changed to improve learning.

Results

Twenty-eight participants attended the workshop at SAEM. Attendees included faculty and resident educators in emergency medicine from multiple institutions in the United States and abroad (e.g., New Zealand, France). The results of the retrospective survey showed significant improvements in self-reported perceptions of participants’ own abilities to (1) recognize how pattern recognition can lead to bias, (2) identify common types of bias in the emergency department, (3) teach trainees about common types of bias, and (4) apply cognitive forcing or debiasing strategies to improve diagnostic reasoning (Table). Effect sizes were calculated and ranged from .57 to .62. Cronbach’s alpha for the pretest was .94 and for the posttest was .82 for all items.

Table 1. Results of Retrospective Evaluation (4-point scale; 1 = Not at All, 2 = A Little, 3 = Moderately, 4 = Very)
Skill
Before
After
Difference
p
Effect size (r)
Please rate your ability for the following skills:
 Recognize how pattern recognition can lead to bias.
2.74
3.67
0.93
<.001
.57
Identify common types of bias in the emergency department.
2.56
3.56
1.00
<.001
.57
 Teach trainees about common types of bias.
1.93
3.04
1.11
<.001
.59
Apply cognitive forcing or “debiasing” strategies to improve diagnostic reasoning.
2.22
3.41
1.19
<.001
.62

In response to the question, “What contributed to your learning from the workshop?” a number of themes appeared in the participant responses. These highlighted the case-based learning, the interactive small groups, the discussion of techniques to mitigate bias, and the take-home pocket card. Similarly, when asked, “What could be changed to improve your learning?” participants suggested more time (“too short a session”), more cases, the inclusion of prereading, more explicit discussion of how to teach these topics, and more concrete ways to educate learners.

Discussion

This resource has the potential to complement the prior learner-focused MedEdPORTAL resources on cognitive bias and debiasing strategies. The workshop was implemented at SAEM in 2017, and retrospective data showed significant improvements in all of the participants’ self-assessed abilities.

Participants’ baseline knowledge was highest in regard to recognizing how pattern recognition can lead to bias, as well as identifying common types of bias encountered in the emergency department, suggesting moderate baseline abilities in these domains. The survey differences were greatest in the domains of teaching trainees about common types of bias and applying debiasing strategies to improve diagnostic reasoning, suggesting that the workshop was particularly successful in providing participants with new skills to educate trainees. The effect sizes were large (>.50) for all domains, demonstrating practical, in addition to statistical, significance. The Cronbach’s alphas for the pretest and posttest were both greater than .70, demonstrating high internal consistency and providing a measure of reliability of the evaluation.

While this workshop can clearly affect self-reported knowledge and skills when delivered in 60 minutes, qualitative participant feedback suggested that a longer (90-minute) time frame may prove more beneficial. This would allow participants to complete four cases and provide a broader exposure to different types of bias. It would also allow more time for explicit discussion of each mitigation strategy as well as how to teach the strategies to different levels of learners. Priming the facilitators to ensure they focus on teaching strategies may also be important.

Providing participants with prereading materials such as the articles by Daniel, Khandelwal, Santen, Malone, and Croskerry9 ; Croskerry11; and Croskerry, Singhal, and Mamede12 would be possible with the appropriate technological infrastructure and participant buy-in. An alternative approach would be to use the workshop as an introduction and then offer more in-depth readings after the session for reinforcement. In local venues, this could be followed up by case discussions from participants’ own experience in clinical practice and perhaps be incorporated into monthly morbidity and mortality conferences. This approach could lead to deeper learning of concepts through spaced repetition.

Of note, several facilitators of this workshop had baseline expertise in cognitive bias and debiasing strategies. The workshop may be more challenging for subject matter novices to implement, and more preparation time to read background literature may be required. That said, the preparation time is unlikely to be prohibitive.

Limitations
Training in cognitive debiasing has not yet been proven to be effective to reduce error,10 though the theoretical basis is sound.12,13 Follow-up empiric studies are needed in the clinical setting to determine if changes in behavior and outcomes are brought about by this training: Do participants and their learners employ cognitive debiasing strategies in real time (not just retrospectively), and what is the ultimate impact on medical error and patient care?

The evaluation forms utilized were developed by the author group, and thus, there is a limitation to their reliability and validity evidence. Furthermore, while the evaluation questions measured the participants’ behaviors as a result of the workshop, these were self-reported behavioral changes. Stronger evidence for the effectiveness of the workshop could be obtained through more rigorous assessment of learners.

Conclusions
Given the potential negative impact that cognitive biases have on patient care, behavior-changing curricula that are effective in mitigating these biases are critically needed. Here, we have presented the pilot implementation of a faculty development workshop that may be able to elicit that change. While this workshop was developed for emergency physicians, the cases could easily be adapted to other specialties to teach trainees about bias and bias-mitigation strategies. Future steps include continual refinement of the session through programmatic evaluation and measurement of higher-order Kirkpatrick behavioral change outcomes18 to test the workshop’s practical effectiveness.


Author Information

  • Michelle Daniel, MD: Assistant Dean of Curriculum, University of Michigan Medical School; Assistant Professor, Department of Emergency Medicine, University of Michigan Medical School; Assistant Professor, Department of Learning Health Sciences, University of Michigan Medical School
  • Michele Carney, MD: Program Director, Pediatric Emergency Medicine Fellowship Program, University of Michigan Medical School; Clinical Assistant Professor, Department of Emergency Medicine, University of Michigan Medical School; Clinical Assistant Professor, Department of Pediatrics and Communicable Diseases, University of Michigan Medical School
  • Sorabh Khandelwal, MD: Residency Program Director, Department of Emergency Medicine, Ohio State University College of Medicine; Samuel Kiehl III MD Professor in Emergency Medicine, Ohio State University College of Medicine
  • Chris Merritt, MD: Residency Program Director, Department of Pediatrics, Alpert Medical School of Brown University; Assistant Professor, Department of Emergency Medicine, Alpert Medical School of Brown University; Assistant Professor, Department of Pediatrics, Alpert Medical School of Brown University
  • Michael Cole, MD: Assistant Professor, Department of Emergency Medicine, University of Michigan Medical School; Co-Director, Clinical Reasoning Course, University of Michigan Medical School
  • Matthew Malone, MD: Resident Physician, Emergency Medicine Residency Program, University of Michigan Medical School
  • Robin R. Hemphill, MD: Chief Patient Safety Officer, Veterans Health Administration; Director, VA National Center for Patient Safety
  • Will Peterson, MD: Resident Physician, Emergency Medicine Residency Program, University of Michigan Medical School
  • John Burkhardt, MD: Assistant Professor, Department of Emergency Medicine, University of Michigan Medical School; Assistant Professor, Department of Learning Health Sciences, University of Michigan Medical School
  • Laura Hopson, MD: Residency Program Director, University of Michigan Medical School; Assistant Professor, Department of Emergency Medicine, University of Michigan Medical School
  • Sally A. Santen, MD, PhD: Assistant Dean for Educational Research and Quality Improvement, University of Michigan Medical School; Professor, Department of Emergency Medicine, University of Michigan Medical School; Professor, Department of Learning Health Sciences, University of Michigan Medical School

Disclosures
None to report.

Funding/Support
None to report.

Ethical Approval
Reported as not applicable.


References

  1. Graber ML. The incidence of diagnostic error in medicine. BMJ Qual Saf. 2013;22(suppl 2):ii21-ii27. https://doi.org/10.1136/bmjqs-2012-001615
  2. Graber M. Diagnostic errors in medicine: a case of neglect. Jt Comm J Qual Patient Saf. 2005;31(2):106-113. https://doi.org/10.1016/S1553-7250(05)31015-4
  3. Croskerry P. A universal model of diagnostic reasoning. Acad Med. 2009;84(8):1022-1028. https://doi.org/10.1097/ACM.0b013e3181ace703
  4. Kahneman D. Thinking, Fast and Slow. New York, NY: Farrar, Straus and Giroux; 2013.
  5. Pelaccia T, Tardif J, Triby E, Charlin B. An analysis of clinical reasoning through a recent and comprehensive approach: the dual-process theory. Med Educ Online. 2011;16(1):5890. https://doi.org/10.3402/meo.v16i0.5890
  6. Birnbaumer D. Avoiding diagnostic error in the ED. Medium website. https://medium.com/@emabstracts/avoiding-diagnostic-error-in-the-ed-979d556cf019. Published April 1, 2016. Accessed June 13, 2017.
  7. Cognitive Bias Codex. Wikimedia Commons website. https://commons.wikimedia.org/wiki/File:Cognitive_Bias_Codex_-_180%2B_biases,_designed_by_John_Manoogian_III_(jm3).jpg. Published September 5, 2016. Updated April 12, 2017. Accessed June 12, 2017.
  8. List of cognitive biases. Wikipedia website. https://en.wikipedia.org/wiki/List_of_cognitive_biases. Accessed June 13, 2017.
  9. Daniel M, Khandelwal S, Santen SA, Malone M, Croskerry P. Cognitive debiasing strategies for the emergency department. AEM Med Educ Train. 2017;1(1):41-42. https://doi.org/10.1002/aet2.10010
  10. Norman GR, Eva KW. Diagnostic error and clinical reasoning. Med Educ. 2010;44(1):94-100. https://doi.org/10.1111/j.1365-2923.2009.03507.x
  11. Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003;78(8):775-780. https://doi.org/10.1097/00001888-200308000-00003
  12. Croskerry P, Singhal G, Mamede S. Cognitive debiasing 1: origins of bias and theory of debiasing. BMJ Qual Saf. 2013;22(suppl 2):ii58-ii64. https://doi.org/10.1136/bmjqs-2012-001712
  13. Croskerry P, Singhal G, Mamede S. Cognitive debiasing 2: impediments to and strategies for change. BMJ Qual Saf. 2013;22(suppl 2):ii65-ii72. https://doi.org/10.1136/bmjqs-2012-001713
  14. Stiegler M, Goldhaber-Fiebert S. Understanding and preventing cognitive errors in healthcare. MedEdPORTAL. 2015;11:10000. https://doi.org/10.15766/mep_2374-8265.10000
  15. Chew KS, van Merrienboer J, Durning S. Teaching cognitive biases in clinical decision making: a case-based discussion. MedEdPORTAL. 2015;11:10138. https://doi.org/10.15766/mep_2374-8265.10138
  16. Ruedinger E, Mathews B, Olson A. Decision - diagnosis: an introduction to diagnostic error and medical decision-making. MedEdPORTAL. 2016;12:10378. https://doi.org/10.15766/mep_2374-8265.10378
  17. Bhanji F, Gottesman R, de Grave W, Steinert Y, Winer LR. The retrospective pre–post: a practical method to evaluate learning from an educational program. Acad Emerg Med. 2012;19(2):189-194. https://doi.org/10.1111/j.1553-2712.2011.01270.x
  18. Kirkpatrick DL. Techniques for evaluating training programs. J Am Soc Train Dir. 1959;13(11):3-9.


Citation

Daniel M, Carney M, Khandelwal S, et al. Cognitive debiasing strategies: a faculty development workshop for clinical teachers in emergency medicine. MedEdPORTAL. 2017;13:10646. https://doi.org/10.15766/mep_2374-8265.10646

Received: June 15, 2017

Accepted: October 10, 2017