Original Publication
Open Access

Teaching Residents Clinical Practice Guidelines Using a Flipped Classroom Model

Published: March 6, 2017 | 10.15766/mep_2374-8265.10548

Appendices

  • Facilitator Guide Presentation OSAS.pptx
  • Facilitator Guide Presentation AOM.pptx
  • Facilitator Guide Presentation ADHD.pptx
  • Room Setup.pptx
  • Guideline Summary Sheet.docx
  • Pre-Posttest.pdf
  • Sample Final Algorithm.pptx
  • Learner Evaluation.docx

All appendices are peer reviewed as integral parts of the Original Publication.

To view all publication components, extract (i.e., unzip) them from the downloaded .zip file.


Abstract

Introduction: Prior studies have demonstrated poor guideline compliance by pediatricians, and there is no published curriculum on how to teach clinical guidelines. Furthermore, in a national survey of pediatric residency training programs conducted in 2015, only two had a formal curriculum for teaching clinical guidelines. This module provides a framework for teaching residents clinical guidelines through a modified flipped classroom approach. Associated materials include a guide for faculty facilitators, sample slides and worksheet, and pictures of the classroom setup. Methods: In this module, the guidelines for acute otitis media (AOM), obstructive sleep apnea syndrome (OSAS), and attention deficit-hyperactivity disorder (ADHD) are taught in three sessions and evaluated with a pre-/posttest assessing knowledge, attitudes, self-efficacy, and satisfaction. Each guideline is delivered in a 30-minute session, with five learners per group. Faculty training requires approximately 30 minutes of preparation. The intervention groups (n = 9 for OSAS, 10 each for AOM and ADHD) received three weekly, half-hour flipped classroom lessons. The control group (n = 19) had no formal guideline education. Results: Pre-/posttests showed a statistically significant improvement in knowledge and attitudes in the group of interns who received this educational intervention over the control group. The learners rated the sessions as highly effective. Discussion: This module provides an efficient and effective way of utilizing a modified flipped classroom approach to teach learners the correct use of clinical guidelines, a skill residents must master to provide evidence-based care. This curriculum has been successfully incorporated into our pediatric residency program.


Educational Objectives

By the end of this module, learners will be able to:

  1. Discuss one key action statement in a clinical practice guideline for acute otitis media (AOM), obstructive sleep apnea syndrome (OSAS), and attention deficit-hyperactivity disorder (ADHD).
  2. Describe one relevant clinical example for their assigned key action statement.
  3. Teach back a minimum of one key point of the key action statement to their small group.
  4. Create a diagnostic or treatment algorithm for the management of AOM, OSAS, and ADHD using relevant clinical practice guidelines.

Figures and Tables

 Table 1. Guideline Action Statements Divided Into Learning Points
Learning Point Key Action Statements
Obstructive Sleep Apnea Syndrome Acute Otitis Media Attention Deficit-Hyperactivity Disorder
1 1 1A 1
2 2A + 2B 1B + 1C 2
3 3 + 4 3A + 3B + 3C + 3D 3
4 5A + 5B 4A + 4B + 4C 5A + 5B + 5C
5 6 + 7 5A + 5B 6
 Table 2. Percentage Score Change on Pre- Versus Posttests
Disorder Control Groupa Intervention Group p Value
Obstructive sleep apnea syndromeb 9.92 20.96 0.03
Acute otitis mediac 4.11 16.13 0.009
Attention deficit-hyperactivity disorderd −5.26 21.05 0.005
an = 19.
bn = 9.
cn = 10.
dn = 10.
Table 3. Resident Likert-Scalea Evaluation of Flipped Classroom Sessions 
Session Facilitation Handouts Comfort Overall
Obstructive sleep apnea syndrome 4.5 4.375 3.875 4.375
Acute otitis media 4.375 4.5 3.75 4.375
Attention deficit-hyperactivity disorder 4.18 4 3.45 4.18
aFive-point Likert scale (1 = poor, 5 = excellent).

Introduction

Studies show that although adherence to and use of clinical practice guidelines vary widely in pediatric practice, these guidelines are generally used less frequently than similar ones in adult medicine. In Flores, Lee, Bauchner, and Kastner’s evaluation of 1,088 pediatricians, practice guidelines were used by only 35%, despite the fact that 89% of respondents agreed that practice guidelines improve outcomes and that a vast majority (94%) desired to improve quality.1 In another study, Forrest and colleagues evaluated the use of a clinical decision support system. Before their intervention, it was found that only 15% of acute otitis media (AOM) and 5% of otitis media with effusion guidelines were followed.2 Rushton, Fant, and Clark described similar results when evaluating the use of the American Academy of Pediatrics (AAP) diagnostic criteria for attention deficit-hyperactivity disorder (ADHD), reporting that only 25.8% of clinicians indicated routine use of all four diagnostic components in the survey.3 Little evidence has been produced and few explanations made as to why pediatricians neither follow nor adhere to clinical practice guidelines. Meanwhile, there is no published curriculum on how to teach clinical guidelines. Furthermore, in a national survey of pediatric residency training programs we conducted in 2015, only two indicated they had a formal curriculum for teaching the guidelines. This is a true gap in resident education, and we hypothesize that if residents are taught how to read and apply the clinical guidelines during their training, they may be more likely to follow them when they go into practice.

While medical educators and physicians struggle to teach and implement clinical practice guidelines, there is a movement to change the way medicine is taught that offers a tangible solution. Medical educators currently advocate moving away from the traditional classroom model of didactics and towards a flipped classroom approach (also called blended learning or an inverted classroom), where the learners engage with the material beforehand (prework) and participate in small active-learning groups during a facilitated session.4-7 The flipped classroom has been shown to be equally as effective, if not more so, in knowledge acquisition across multiple settings (e.g., medical school core courses such as biochemistry and core clerkships like surgery).8-12 Additionally, learners tend to prefer this model of self-directed learning over the traditional classroom.12,13

Residency training, however, poses a challenge to the flipped classroom model due to the intense clinical demands on trainees that limit their ability and willingness to complete the required prework.14 In an effort to ensure pediatric residents are learning essential clinical practice guidelines during their training, we propose a hybrid flipped classroom model that incorporates the prework directly into the small active-learning groups. Because the prework is built into the facilitated session, residents are able to briefly engage with the material on their own and then work in a small group to apply, create, and analyze the clinical practice guidelines to develop their own diagnostic or treatment algorithm for the condition.

Methods

Hybrid Flipped Classroom Intervention Development
Guideline selection: We developed a pilot study to evaluate the effectiveness of three 30-minute hybrid flipped classrooms to teach pediatric interns three clinical practice guidelines. The clinical practice guidelines selected for this study were those for AOM, obstructive sleep apnea syndrome (OSAS), and ADHD.15-17 Through expert consensus, we distilled five key learning points for each guideline from the guidelines’ key action statements (Table 1). This was done by having three of the authors independently review the three guidelines and select the most pertinent key action statements to teach in each session. Mediation by the senior author was used to address discrepancies. These learning points were then made the focus of the modified flipped classroom sessions.

 Table 1. Guideline Action Statements Divided Into Learning Points
Learning Point Key Action Statements
Obstructive Sleep Apnea Syndrome Acute Otitis Media Attention Deficit-Hyperactivity Disorder
1 1 1A 1
2 2A + 2B 1B + 1C 2
3 3 + 4 3A + 3B + 3C + 3D 3
4 5A + 5B 4A + 4B + 4C 5A + 5B + 5C
5 6 + 7 5A + 5B 6

Intervention design: The intervention groups (n = 9 for OSAS, 10 each for AOM and ADHD) received weekly, half-hour flipped classroom lessons for the three guidelines, while the control group (n = 19) had no formal education about guidelines. The intervention consisted of a 30-minute structured small-group activity led by a facilitator. The facilitator led the session with a simple PowerPoint (Appendix A for OSAS, Appendix B for AOM, or Appendix C for ADHD) to advance the groups through each portion of the intervention.

Intervention Implementation Guide
Classroom setup: Prior to the session, the room is arranged with tables and chairs forming groups of five, with a printed copy of the clinical practice guidelines of that session’s selected topic (OSAS, AOM, or ADHD) at each seat, as demonstrated in Appendix D.15-17 Each printed copy of the clinical practice guideline has a post-it note with an assigned learning point derived from the key action statements (e.g., Learning Point 1 for Group Member 1, Learning Point 2 for Group Member 2, etc.). As stated previously, the five assigned learning points were identified earlier through expert consensus and reflect the content on the pre/post assessment test. Each small group receives one guideline summary sheet (Appendix E), a piece of chart paper, and markers.

Introduction and objectives: All three sessions begin with the facilitator reviewing the learning objectives and introducing the clinical practice guideline for the session using the presentation found in Appendix A, B, or C. Facilitators should conduct the presentation only through slide seven for Appendix A, slide six for Appendix B, and slide seven for Appendix C. The remaining slides for each presentation are used later in the session. Typically, the facilitator should set aside 2 minutes for this session. Please note that during the students’ initial session, the pretest (Appendix F) is administered first to assess student presession knowledge of clinical guidelines for OSAS, AOM, and ADHD. The facilitator should allow an additional 5-10 minutes to administer this pretest during the first session.

Prework: Residents are instructed to quietly and individually read their assigned learning point. They are prompted to identify the main point they think is most relevant for all pediatric residents to know and develop a relevant clinical example. This is the prework portion of the flipped classroom model that we adapted to occur during our hybrid sessions. The facilitator allots 3 minutes for this portion.

Teach back: The facilitator assigns 12 minutes for this portion. Residents are asked to teach back the main point of their assigned learning point to their group in approximately 2 minutes (in sequential order of learning guideline). A scribe is assigned to record the main points for the group on the guideline summary sheet (Appendix E). The residents are encouraged to highlight key tables, figures, or charts while explaining their relevance. This generates small-group discussion as the remaining group members can ask clarifying questions and compare or contrast their prior knowledge to the action statement. The process is repeated roughly every 2 minutes until each group member has taught his or her assigned learning point to the small group.

Create diagnostic flowchart/algorithm: During the remaining 10 minutes of the session, each group compiles the content gathered on the guideline summary sheet (Appendix E) into a diagnostic and treatment algorithm or flowchart, a sample of which can be found in Appendix G. Additional examples are found in Appendices A-C, and these may be revisited if needed. The facilitators can circulate the room but should refrain from interjecting unless a group is off task and needs to be redirected. The purpose of this time-constrained portion is to challenge group members to apply the knowledge they have just learned to create a tool they can reference in the future. If the group needs prompting, possible questions can include “How will you use this information next time you are in clinic?”, “What component was most challenging or interesting?”, and “What is the first step to diagnosing this condition?” The end products may vary in their design and structure (e.g., a flowchart vs. a table vs. a diagram), but the content should reflect the learning points.

Check for understanding—apply to case: The facilitator asks the large group a series of clinical questions that each small group must answer using its diagnostic flowchart/algorithm. These questions are provided in the facilitator PowerPoints and begin on slide eight of Appendix A, slide seven of Appendix B, and slide eight of Appendix C. This allows the facilitator to reinforce the main learning points and find any major gaps in the flowcharts/algorithms before the end of the session. Any misinformation can be quickly corrected and retaught to ensure the residents do not leave with incorrect information. Three minutes are generally allotted for this component. If the students are on their final session, they are given the posttest evaluation (Appendix F). Students should be given an additional 5-10 minutes to complete the posttest.

Results

A deidentified 32-question test (Appendix F) was developed to assess pediatric interns’ knowledge, attitudes, and general use of clinical practice guidelines both prior to and after they had experienced the three sessions. For the purpose of the pilot study, the test was administered to 29 pediatric interns (PGY1) during Week 1 (preintervention) and repeated on Week 4 (postintervention). Ten interns received the intervention, and 19 received the control (no-guidelines curriculum). A single investigator graded all tests using a scoring rubric. Evaluation of the quality of the facilitation, content, and structure of the three flipped classroom sessions was obtained at Week 4 (postintervention) using Likert scales and open-ended feedback for improvement (Appendix H). The study was Stanford IRB exempt.

All of our pediatric interns (N = 29, 22 female, seven male) were included in the study. Descriptive statistics and unpaired, two-tailed t tests (p < .05) were used for analyses. Before the intervention, three out of 29 interns indicated on the pretest (Appendix F) that they had received teaching on OSAS, 13 out of 29 on AOM, and eight out of 29 on ADHD, respectively. Only 20 of 29 interns knew there were any clinical practice guidelines published by the AAP, and these 20 were able to name on average only three of the 10 guidelines. On a 5-point Likert scale (1 = very important, 5 = not important), 28 interns rated the importance of a pediatrician knowing and using the guidelines as very important; the other intern rated these as somewhat important.

The interns had an average knowledge score of 44% on the pretest. The score breakdown by guideline was 45% for OSAS, 11% for AOM, and 79% for ADHD. A convenience sample was used to select the intervention group based on the interns who were at the main training site (Lucile Packard Children’s Hospital). The number of interns in each intervention group was nine for OSAS, 10 for AOM, and 10 for ADHD.

When the posttest was adminsistered, the intervention interns had a statistically significant increase in knowledge over the control interns: 21% versus 10% for OSAS (p = .03), 16% versus 4% for AOM (p = .009), and 21% versus −5% for ADHD (p = .005; Table 2). There was a statistically significant increase in self-efficacy posttest versus pretest but no difference between intervention and control groups.

 Table 2. Percentage Score Change on Pre- Versus Posttests
Disorder Control Groupa Intervention Group p Value
Obstructive sleep apnea syndromeb 9.92 20.96 0.03
Acute otitis mediac 4.11 16.13 0.009
Attention deficit-hyperactivity disorderd −5.26 21.05 0.005
an = 19.
bn = 9.
cn = 10.
dn = 10.

All participants filled out anonymous evaluations (Appendix H) after the session to assess facilitation, handouts, comfort using the guideline, and overall session. A 5-point Likert scale (1 = poor, 5 = excellent) was used. Overall, the sessions were rated 4.4 for OSAS, 4.4 for AOM, and 4.2 for ADHD. The sessions received unanimously high marks for all categories (facilitation, handouts, and comfort; Table 3).

Table 3. Resident Likert-Scalea Evaluation of Flipped Classroom Sessions 
Session Facilitation Handouts Comfort Overall
Obstructive sleep apnea syndrome 4.5 4.375 3.875 4.375
Acute otitis media 4.375 4.5 3.75 4.375
Attention deficit-hyperactivity disorder 4.18 4 3.45 4.18
aFive-point Likert scale (1 = poor, 5 = excellent).

Discussion

We developed and implemented a modified flipped classroom approach to teach clinical guidelines, which successfully improved intern knowledge and attitudes when compared with standard education. These sessions were well received. In fact, our PGY2 and PGY3 residents insisted the sessions be taught to all levels of learners. Furthermore, it was noticed that the interns were incorporating the guideline knowledge into their actual patient care in clinic and on the wards and at times knew the guidelines better than the PGY2 and PGY3 residents. This created further interest among the PGY2 and PGY3 residents, which led to us offering this module to all levels in later months. The curriculum was expanded to include all 10 published AAP guidelines, and it is now a core component of our primary care conference series. In addition, Drs. Peterson, Louden, and Gribben presented this work at both the Association of Pediatric Program Directors Chief Resident Forum and the Academic Pediatric Association Education Committee’s national meeting in 2016.

Although this curriculum was created to teach clinical guidelines, the teaching methods can also be adapted to teach any topic to medical students, residents, and fellows. This work successfully showed that a modified flipped classroom approach can be used for busy clinical learners who might not have sufficient time to prepare ahead of class. The interactive nature of the sessions allowed learners to solidify their baseline knowledge and apply it to real clinical scenarios. In this way, the key concepts of the clinical guideline were reinforced at least four times throughout the session.

A few revisions have been made to the curriculum since inception. These include the creation of advanced and modified sessions with slight variations. For example, in the hyperbilirubinemia guidelines, each group works on one action statement and reports out (a jigsaw technique).18 For AOM, skills stations have been added as a form of case-based application, and as the guidelines are discussed, residents match the category of tympanic membrane disease with pictures of tympanic membranes. Regarding challenges in implementing the curriculum, although there was initially student discomfort with learning via a new model and some uncertainty as to the appropriate level of participation, the students quickly learned how to participate. In addition, given the short duration of our morning report (30 minutes), we found that the facilitator needs to stick closely to the time line so as not to get off schedule.

Despite the curriculum’s rigorous design and implementation, a few limitations do exist. A convenience sample of interns at a particular site was studied, and although the curriculum was required of all interns at that site, residents could self-select into receiving or not receiving it. Moreover, this was a small pilot study at a single institution. Follow-up studies should investigate the sustainability of the increased knowledge and attitudes. Our educational intervention provides a robust framework for teaching how to use clinical guidelines and demonstrates an interactive teaching method (the modified flipped classroom) that is a more feasible construct than the true flipped classroom for busy clinical learners.


Author Information

  • Jaime Peterson, MD: Postdoctoral Fellow, Stanford University School of Medicine; Clinical Instructor in General Pediatrics, Stanford University School of Medicine
  • Taylor Louden, MD: Postdoctoral Fellow in Pediatric Emergency Medicine, Seattle Children's Hospital
  • Valerie Gribben, MD: Clinical Instructor in Pediatrics, Division of Hospitalist Medicine, Washington University in St. Louis School of Medicine
  • Rebecca Blankenburg, MD, MPH: Clinical Associate Professor, Stanford University School of Medicine; Pediatric Residency Program Director, Stanford University School of Medicine; Associate Chair of Education, Stanford University School of Medicine

Disclosures
None to report.

Funding/Support
None to report.

Prior Presentations
Peterson J, Louden DT, Gribben V, Blankenburg R. A “flipped classroom” model: teaching interns clinical guidelines. Poster presented at: Pediatric Academic Societies Annual Meeting; April 30-May 3, 2016; Baltimore, MD.

Peterson J, Louden DT, Gribben V, Blankenburg R. A “flipped classroom” model: teaching interns clinical guidelines. Poster presented at: Lucile Packard Children’s Hospital Stanford Research Day; April 2016; Palo Alto, CA.

Peterson J, Louden DT, Gribben V, Blankenburg R. A “flipped classroom” model: teaching interns clinical guidelines. Poster presented at: Association of Pediatric Program Directors Annual Spring Meeting; March 30-April 2, 2016; New Orleans, LA.

Peterson J, Louden DT, Gribben V, Blankenburg R. Teaching residents clinical practice guidelines using a flipped classroom. Presented at: Pediatric Grand Rounds at the Lucile Packard Children’s Hospital; June 2016; Palo Alto, CA.

Peterson J, Louden DT, Gribben V, Blankenburg R. Teaching residents clinical practice guidelines using a flipped classroom. Presented at: Pediatric Grand Rounds at the Santa Clara Valley Medical Center; June 2016; San Jose, CA.

Peterson J, Louden DT, Gribben V, Blankenburg R. A “flipped classroom” model: teaching interns clinical guidelines. Presented at: APA Education Committee Meeting at the Pediatric Academic Societies Annual Meeting; April 30-May 3, 2016; Baltimore, MD.

Peterson J, Louden DT, Gribben V, Blankenburg R. A “flipped classroom” model: teaching interns clinical guidelines. Presented at: Chief Resident Forum at the Association of Pediatric Program Directors Annual Spring Meeting; March 30-April 2, 2016; New Orleans, LA.

Ethical Approval
Reported as not applicable.


References

  1. Flores G, Lee M, Bauchner H, Kastner B. Pediatricians’ attitudes, beliefs, and practices regarding clinical practice guidelines: a national survey. Pediatrics. 2000;105(3):496-501.
  2. Forrest CB, Fiks AG, Bailey LC, et al. Improving adherence to otitis media guidelines with clinical decision support and physician feedback. Pediatrics. 2013;131(4):e1071-e1081. https://doi.org/10.1542/peds.2012-1988
  3. Rushton JL, Fant KE, Clark SJ. Use of practice guidelines in the primary care of children with attention-deficit/hyperactivity disorder. Pediatrics. 2004;114(1):e23-e28. https://doi.org/10.1542/peds.114.1.e23
  4. Prober CG, Khan S. Medical education reimagined: a call to action. Acad Med. 2013;88(10):1407-1410. https://doi.org/10.1097/ACM.0b013e3182a368bd
  5. Vogel L. Educators propose “flipping” medical training. CMAJ. 2012;184(12):E625-E626. https://doi.org/10.1503/cmaj.109-4212
  6. Tolks D, Schäfer C, Raupach T, et al. An introduction to the inverted/flipped classroom model in education and advanced training in medicine and in the healthcare professions. GMS J Med Educ. 2016;33(3):Doc46. https://doi.org/10.3205/zma001045
  7. Bergmann J, Sams A. Flip Your Classroom: Reach Every Student in Every Class Every Day. Arlington, VA: International Society for Technology in Education; 2012.
  8. Prober CG, Heath C. Lecture halls without lectures—a proposal for medical education. N Engl J Med. 2012;366(18):1657-1659. https://doi.org/10.1056/NEJMp1202451
  9. Morton DA, Colbert-Getz JM. Measuring the impact of the flipped anatomy classroom: the importance of categorizing an assessment by Bloom’s taxonomy [published online ahead of print July 18, 2016]. Anat Sci Educ. https://doi.org/10.1002/ase.1635
  10. Liebert CA, Mazer L, Merrell SB, Lin DT, Lau JN. Student perceptions of a simulation-based flipped classroom for the surgery clerkship: a mixed-methods study. Surgery. 2016;160(3):591-598. https://doi.org/10.1016/j.surg.2016.03.034
  11. U.S. Department of Education, Office of Planning, Evaluation, and Policy Development. Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies. Washington, DC: U.S. Department of Education; 2010.
  12. McLaughlin JE, Roth MT, Glatt DM, et al. The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Acad Med. 2014;89(2):236-243. https://doi.org/10.1097/ACM.0000000000000086
  13. Tan E, Brainard A, Larkin GL. Acceptability of the flipped classroom approach for in-house teaching in emergency medicine. Emerg Med Australas. 2015;27(5):453-459. https://doi.org/10.1111/1742-6723.12454
  14. Tainter CR, Wong NL, Cudemus-Deseda GA, Bittner EA. The “flipped classroom” model for teaching in the intensive care unit: rationale, practical considerations, and an example of successful implementation. J Intensive Care Med. 2017;32(3):187-196. https://doi.org/10.1177/0885066616632156
  15. Lieberthal AS, Carroll AE, Chonmaitree T, et al. The diagnosis and management of acute otitis media. Pediatrics. 2013;131(3):e964-e999. https://doi.org/10.1542/peds.2012-3488
  16. Marcus CL, Brooks LJ, Draper KA, et al. Diagnosis and management of childhood obstructive sleep apnea syndrome. Pediatrics. 2012;130(3):576-584. https://doi.org/10.1542/peds.2012-1672
  17. Wolraich ML. The new attention deficit hyperactivity disorder clinical practice guidelines published by the American Academy of Pediatrics. J Dev Behav Pediatr. 2012;33(1):76-77. https://doi.org/10.1097/DBP.0b013e318241eadf
  18. Aronson E, Blaney N, Stephin C, Sikes J, Snapp M. The Jigsaw Classroom. Beverly Hills, CA: Sage Publishing; 1978.


Citation

Peterson J, Louden DT, Gribben V, Blankenburg R. Teaching residents clinical practice guidelines using a flipped classroom model. MedEdPORTAL. 2017;13:10549. https://doi.org/10.15766/mep_2374-8265.10549

Received: October 5, 2016

Accepted: January 20, 2017