Original Publication
Open Access

A Multicomponent Intervention to Improve Pneumococcal Vaccination Knowledge Among Internal Medicine Residents

Published: June 10, 2016 | 10.15766/mep_2374-8265.10414


  • Improving Pneumococcal Vaccination Rates in Clinic Video.mp4
  • Pocket Cards.pdf
  • Quiz Questions-Blank.pdf
  • Quiz Questions-Answers.pdf
  • Vaccine Indication Table.pdf
  • Post Education Survey.pdf

All appendices are peer reviewed as integral parts of the Original Publication.

To view all publication components, extract (i.e., unzip) them from the downloaded .zip file.


Introduction: The Advisory Committee on Immunization Practices updated its pneumococcal vaccination guidelines in September 2014 and provided an additional recommendation in February 2016. We perceived a knowledge gap related to these guidelines among residents who serve as primary care physicians during postgraduate training. Our research team confirmed the presence of this knowledge gap and designed a curriculum focused on the current guidelines for pneumococcal vaccination. Methods: This curriculum consists of a preeducation quiz and survey, as well as an educational video, pocket card, and poster. The educational materials were then disseminated over a 7-week period and included a short video, handouts (laminated pocket cards), and summaries of guideline recommendations in electronic format. Results: The quiz, which includes eight clinical vignette selected-response items, revealed a knowledge deficiency. For example, only a minority of residents (31.2%) correctly chose the appropriate pneumococcal vaccination schedule for an elderly patient with multiple comorbid and chronic medical ailments. A postintervention survey showed that a majority of residents (87.5%) found the educational tools effective in improving understanding and implementation of vaccine guidelines. Discussion: This novel educational strategy is designed to increase resident knowledge of pneumococcal vaccination guidelines with eventual translation to actual clinical practice.

Educational Objectives

By the end of this module, the learner will be able to:

  1. Determine knowledge, attitudes, and actions regarding the updated pneumococcal vaccination guidelines through a quiz and survey.
  2. Recall accurate information from the 2014 Advisory Committee on Immunization Practices Guidelines for Adult Pneumococcal Vaccination.
  3. Analyze clinical vignettes and select the appropriate pneumococcal vaccination strategy.


Invasive pneumococcal disease is a well-recognized cause of morbidity and mortality. In the United States, the disease incidence for adults 65 years of age or greater is approximately 37 cases per 100,000.1 This corresponds to a death rate of 5.61 deaths per 100,000 for elder adults.1 Vaccines that reduce the risk of pneumococcal disease by conferring protection against Streptococcus pneumoniae are available but are underutilized. In fact, national vaccination rates are historically low, demonstrating a serious practice gap among providers. Recent estimates show vaccination rates of 54.4% for adults greater than 65 years of age.2 Vaccination coverage for United States adults ages 18-64 years with high-risk medical conditions has been estimated to be as low as 21.0% overall.2 Internal medicine residents serve as primary care physicians and have a role to play in ensuring appropriate vaccinations are offered to patients, including vaccines against S. pneumoniae. The internal medicine residency program at our academic medical center does not have a curriculum that specifically addresses vaccination indications and guidelines. Increased knowledge and understanding of pneumococcal vaccination guidelines are an important aspect of postgraduate training that may lead to improvement in clinical practice. We created these educational materials as part of a study to improve knowledge and implementation of pneumococcal vaccination guidelines.


The design of instructional materials was based on constructivist learning theory, which views learning as an active internal mental process of building meaning from the environment. In this theory, the role of the educator is to foster critical reflection and provide a framework that allows the learner to construct meaningful connections between new and previously learned information. Therefore, the components of our intervention can be viewed as those that promote critical reflection (quiz and narrated guidelines summary) and those that promote construction of new learning (handout/pocket card summary). The delivery of these instructional materials was guided by self-directed learning theory and allowed learners to engage with the material at a time and frequency best suited for their individual learning.

Each categorical internal medicine resident (PGY-I–PGY-III) at an academic medical center was invited to complete an optional and anonymous quiz and survey. The quiz (Appendices C & D) was constructed by the study team and consisted of eight clinical vignette selected-response items with four discrete answer choices. The topics of evaluation included complications of pneumococcal disease, vaccination schedules for adult patients either younger than 65 years of age or older, and vaccine components.

The survey, also constructed by the study team, consisted of 10 questions that assess the resident’s reported knowledge, attitude, and action toward the updated pneumococcal vaccination guidelines. The answer choices are a five-item Likert scale ranging from strongly disagree to strongly agree.

We allowed the residents approximately 2 weeks to complete the optional and anonymous quiz and survey. Following data collection, we started a 7-week curriculum to enhance understanding of the pneumonia vaccine guidelines. To maximize efficiency and implement principles of self-directed and just-in-time learning, a traditional didactic session regarding pneumococcal vaccination guidelines was not utilized. Rather, the study team applied shorter and more directed forms of education. For example, the study team created an annotated video (7 minutes in duration; Appendix A) highlighting vaccination guidelines; this was e-mailed to the residents. Furthermore, the study team distributed handouts (pocket cards; Appendix B) consisting of a simplified vaccination algorithm. This same flowchart of information was converted to an electronic format and e-mailed to the residents; it was intended for use on a smart phone. Lastly, the study team created a vaccine indication poster (Appendix E) based upon guidelines from the Centers for Disease Control and Prevention (CDC) that was used as a point-of-care reminder in clinic work rooms. Using a combination of these educational modalities, the study team also sent out short e-mails and delivered chalk talks to the residents to discuss salient aspects of the updated pneumococcal guidelines. The chalk talks were based upon clinical vignettes, often using actual patients in the clinic, to emphasize the appropriate vaccination schedule based upon age, risk factors, and immunization history. The video narration, pocket card distribution, point-of-care reminder poster, and chalk talks were all performed by residents on our study team in a peer-teaching role.

Given the erratic nature of resident work schedules, we delivered a small yet practical weekly educational intervention for 7 weeks. Below is an approximate time frame for the educational interventions.

Educational Intervention Schedule

  • Week 1: Distribute handouts (pocket cards) to the residents.
  • Week 2: Send web link to the annotated video via e-mail to the residents.
  • Week 3: Display the vaccination indication table from the CDC in the resident workroom.
  • Week 4: Employ brief chalk talks in the resident workroom using the pocket card algorithm.
  • Week 5: Send residents an e-mail containing the vaccination guidelines from the CDC.
  • Week 6: Send residents an e-mail to encourage vaccination practice and acknowledge their important role.
  • Week 7: Resend web link to the annotated video via e-mail to the residents.

Finally, the residents were invited to complete an optional and anonymous posteducation survey (Appendix F). This electronic survey, which was created by the study team, was sent to all of the residents after 7 weeks of educational intervention. This survey consisted of one item inquiring whether the educational tools were effective in improving understanding and implementation of current pneumococcal vaccine guidelines for adults. The answer choices were a five-item Likert scale ranging from strongly disagree to strongly agree. There was also a section that allowed residents to free-text any comments regarding the effectiveness of the vaccine education project.


The precurriculum quiz and survey were distributed in person to 101 internal medicine residents in the fall of 2015. Our response rate was 80% (n = 81) for the quiz and 78% (n = 79) for the survey.

With regard to the precurriculum survey, 90% of the residents either agreed or strongly agreed that they would benefit from additional education concerning the current guidelines for vaccination. Furthermore, 76% of the residents either agreed or strongly agreed that more clinic patients would receive pneumonia vaccines with a better understanding of the updated guidelines. Additionally, only 32% of residents strongly agreed that clinic preceptors had a satisfactory understanding of the current pneumococcal vaccination guidelines.

Overall, residents believed that our educational intervention was useful and effective. The response rate was 63% (n = 64) for the e-mailed posteducation survey. Among respondents, 87.5% (n = 56) agreed or strongly agreed the educational tools were effective in improving understanding and implementation of current pneumococcal vaccine guidelines for adults. Furthermore, the residents expressed positivity and enthusiasm regarding the educational curriculum through the free-text comment section of the survey. These comments were analyzed qualitatively for emergent themes, which are summarized below.

Helpful for Clinical Practice

  • “The quick reference card was very useful in the busy clinic setting—great idea.”
  • “The pocket card is extremely helpful. I use it regularly to make sure my patients have the appropriate vaccinations!”
  • “The cards are very helpful! I’ve seen multiple residents use them and it has helped us ensure our patients are vaccinated.”
  • “I now get excited when I see [a patient] that has not gotten [the vaccine] because I am doing something that’s helpful. This initiative really helped me see my deficiency and also improve . . .  patient care.”

Effective for Learning

  • “This was great. I feel like I have a much better understanding of the vaccines. And when I do forget something, my pocket card is right there to bail me out. Something like this should be implemented at other programs as well.”
  • “The guidelines are hard to understand, but the card is always a quick resource to remember the proper guidelines.”
  • “This greatly helped to simplify an otherwise convoluted guideline.”


  • “I use the posters in clinic daily to ensure my patients are up to date on their pneumococcal vaccination. Easily one of the most practice changing tools for my practice as a [primary care physician].”
  • “I always forgot a couple of the qualifiers for people under the age of 65, but this raised my awareness and now I think about it with all of my patients, as they usually qualify.”
  • “Extremely effective intervention. Pocket cards were well utilized and have greatly improved the vaccination status of my patients.”
  • “This pocket card and teaching . . . literally changed my practice. I have been vaccinating much more since you piloted this project.”

The precurriculum quiz was analyzed for additional information on its validity and the presence of a knowledge gap. On average, residents answered 4.37 of the eight items correctly on the quiz, with a standard deviation of 1.60. Table 1 shows a breakdown based upon year of training (PGY-I–PGY-III). Quiz scores tended to correlate with level of training.

Table 1. Quiz Performance by Training Level 
PGY Level
Average % Items Correct

A test-item analysis demonstrated the eight-item quiz was able to discriminate between high and lower performers. The point-biserial correlation (rpbis) is a measure of item discrimination; it reveals the correlation between score on an individual item and score on the quiz overall. A positive rpbis indicates the residents who scored highly on the quiz answered the item correctly. In general, point-biserial correlations of .45 to .65 are considered very high.3,4 The average rpbis was .41; the values for each item are shown in Table 2. In addition, the average item difficulty was .55.

 Table 2. Statistical Analysis for the Eight-Item Precurriculum Quiz
Pneumococcal disease complication
Vaccine schedule for healthy elderly patient 
Vaccine schedule for elderly patient with chronic diseases
Combining influenza and PPSV-23 vaccines
Vaccine schedule for younger patient with chronic diseases
Vaccine schedule for elderly patient with chronic diseases
Vaccine schedule for recently diagnosed HIV patient
Vaccine classification for PCV-13
Abbreviation: rpbis, point-biserial correlation.


Based upon our investigation, a gap exists in the current resident curriculum to address vaccination practice. The survey responses show that residents desire a clearer explanation of the seemingly complex pneumococcal guidelines. In addition, the preeducation quiz results suggest a learning deficit is present. For example, in a clinical vignette that described an elderly patient with chronic medical conditions, only 31.2% of the residents chose the correct response. Our multicomponent educational tools, which include a brief annotated video, handouts (pocket cards), and point-of-care reminders, are designed to fit into the fast-paced learning environment in current postgraduate education. Additionally, we believe this project shows the effectiveness of peer-to-peer education. While faculty input was critical for the project, the residents on the research team directly instructed their colleagues and implemented the teaching modalities. This project demonstrates how novel teaching methods can supplement the traditional didactic learning format. We believe that peer-to-peer education can be extremely effective. The posteducation survey reveals that the vast majority of the residents found the educational tools helpful and effective. In fact, multiple residents commented upon the practice-changing nature of a simple pocket card that distills complicated vaccination guidelines.

There were numerous insights gained from this investigation. Given the erratic nature of resident work hours, the research team encountered difficulty obtaining a satisfactory response rate for the quiz and survey. Simply obtaining these assessments from the residents was perhaps the most time-consuming and challenging aspect of the project. It is often recognized that inpatient curriculum overshadows outpatient training during residency. We found this to be true at our academic medical center and believe our educational curriculum helps to address this gap. Additional education focused on preventative medicine, specifically, vaccinations, may help in training a well-balanced internist.

To gain more insight into the effectiveness of these materials, we plan to redistribute the quiz and the survey to the residents. In a future study, we plan to examine the actual rates of pneumococcal vaccination for patients under the care of resident physicians. This project would help assess the relationship between attitude and actual clinical practice.

Author Information

  • Matthew W. Wilson, MD: Resident Physician, Wake Forest Baptist Medical Center; Resident Physician, Wake Forest School of Medicine; Resident Physician, United States Navy
  • Blair J. Brown, MD: Resident Physician, Wake Forest School of Medicine; Resident Physician, Wake Forest Baptist Medical Center
  • Matthew C. Miles, MD, MEd: Assistant Professor of Pulmonary, Critical Care, Allergy and Immunologic Disease, Wake Forest School of Medicine; Assistant Professor of Pulmonary, Critical Care, Allergy and Immunologic Disease, Wake Forest Baptist Medical Center

None to report.

None to report.

Ethical Approval
This publication contains data obtained from human subjects and received ethical approval.


  1. Cox CM, Link-Gelles R. Pneumococcal. In: Roush S, Baldy L, eds. Manual for the Surveillance of Vaccine-Preventable Diseases. Atlanta, GA: Centers for Disease Control and Prevention; 2008.
  2. National Center for Health Statistics. Health, United States, 2012: With Special Feature on Emergency Care. Hyattsville, MD: National Center for Health Statistics; 2013.
  3. Downing SM, Yudkowsky R. Assessment in Health Professions Education. New York, NY: Routledge; 2009.
  4. McGahee TW, Ball J. How to read and really use an item analysis. Nurse Educ. 2009;34(4):166-171. http://dx.doi.org/10.1097/NNE.0b013e3181aaba94


Wilson MW, Brown BJ, Miles MC. A multicomponent intervention to improve pneumococcal vaccination knowledge among internal medicine residents. MedEdPORTAL. 2016;12:10414. https://doi.org/10.15766/mep_2374-8265.10414

Received: November 22, 2015

Accepted: May 12, 2016