Curriculum to Develop Documentation Proficiency Among Medical Students in an Emergency Medicine Clerkship
By the end of this activity, learners will be able to:
Describe the components required for a complete emergency department patient note.
Independently compose a billable patient note that requires little to no alteration by faculty physicians based on a simulated patient encounter.
Independently compose billable patient notes that require little to no alteration by faculty physicians based on real-life emergency department patient encounters.
Articulate the emergency differential diagnosis for a patient presenting to the emergency department with chest pain.
Medical student documentation in the electronic health record (EHR) is a controversial issue among physician educators. Although there are concerns among clerkship directors about decreased teaching time due to the use of EHRs, there are also concerns that not allowing students to document in the EHR would hinder them from fully engaging in patient care.1 A collaborative statement released by the multispecialty Alliance for Clinical Education encourages medical student documentation in the patient record as well as robust review and assessment of student notes by medical educators.2 Nonetheless, a 2016 survey of emergency medicine (EM) clerkship directors revealed myriad barriers to allowing medical students to document in the EHR, mostly rooted in administrative and medicolegal concerns.3
More recently, the Association of American Medical Colleges (AAMC) included documenting a clinical encounter in the patient record among its core Entrustable Professional Activities for entering residency.4 In March of 2018, the Centers for Medicare and Medicaid Services (CMS) announced that medical student documentation of patient care services would be accepted for billing purposes.5 This significant change in CMS policy created an additional opportunity and impetus for medical students to be taught how to document proficiently.
The purpose of the current educational resource was to train third- and fourth-year medical students on how to proficiently and independently document an emergency department (ED) patient encounter in the EHR. Previous research on patient notes written by medical students without dedicated documentation training has demonstrated a high rate of inadequate information.6 We conducted a needs assessment, which illustrated the dearth of training that our medical students had received elsewhere throughout medical school. Before being introduced to our curriculum, two-thirds of our learners had received less than 30 minutes of instruction on writing a high-quality ED note, and 86% had received less than 30 minutes of instruction about the billing and coding implications of a patient note.
This curriculum utilized multiple modalities of instruction that were mapped against the four stages of Kolb's experiential learning cycle: concrete experience, reflective observation, abstract conceptualization, and active experimentation.7 Participation in a didactic session and review of reference materials represented abstract conceptualization, writing a sample note based on a standardized patient (SP) encounter fulfilled active experimentation, documenting real clinical encounters in the ED provided concrete experience, and provision of timely feedback on these first several patient notes allowed for reflective observation. By designing this resource around Kolb's learning cycle, we aimed to optimize the educational experience for medical student learners regarding a topic on which they had received sparse training elsewhere in medical school.
Although many principles of effective documentation are shared with other medical specialties and clinical environments, there are also unique aspects of documenting an ED patient encounter that warrant more focused and specific instruction. These aspects include time constraints that benefit from pertinence and succinctness, the frequency with which medical decisions are made based on limited information, the role of an ED visit as a complete and isolated health care encounter, and EM-specific billing requirements.8
Several prior publications in MedEdPORTAL have sought to support the training of resident physicians on documentation skills in specialties aside from EM,9–11 and at least one resource is designed specifically for EM residents documenting clinical encounters in the ED.12 Two other publications broadly address medical student utilization of the EHR,13,14 of which writing patient notes is one of many components. Resources also exist for medical students to learn about documentation in the ED,8 although data on effectiveness or educational outcomes are lacking.
By integrating concepts from and building upon prior published resources, we sought to create a comprehensive multimodal documentation curriculum designed uniquely for medical students. The primary goal of the curriculum was to build proficiency specifically in EM documentation, while also introducing documentation principles broadly applicable across other clinical environments. As an orientation day activity at the beginning of an EM clerkship, the SP activity had a secondary objective of helping students to gain familiarity with the emergency differential diagnosis and diagnostic workup of a patient presenting to the ED with chest pain.
The target audience for this learning resource was third- and fourth-year medical students about to start participating in an EM clerkship. The ideal instructor was an EM attending, fellow, or resident physician with experience in both evaluating and treating ED patients as well as in documenting an EM clinical encounter.
This curriculum was implemented for all third- and fourth-year medical students participating in a 4-week EM elective clerkship at our institution. All medical student participants had previously completed at least 12 months of clinical rotations. Ninety minutes were reserved for documentation training on the first day of the clerkship prior to students taking part in any ED clinical shifts. One educator conducted the instruction for each cohort of eight to 10 students.
During the first 30 minutes, the instructor delivered a group didactic about written documentation concepts to all eight to 10 students according to the facilitator guide (Appendix A). A whiteboard and dry-erase markers were used to write out salient points during the session. The instructor also facilitated student review of a documentation reference card (Appendix B) and sample ED note (Appendix C) during this portion.
Afterwards, all students participated in a formative SP activity as outlined in the facilitator guide (Appendix A). This activity took place in an SP center with simulated exam rooms and stethoscopes available, and SPs were recruited among candidates with prior SP experience who fit the profile of the simulated patient. SPs were provided with the SP script (Appendix D) at least 1 week before the session for home study. Several days prior to the session, an SP educator met with the SPs for 15–30 minutes to review the case and rehearse responses by playing the role of a learner. During the session, two SPs were stationed in two separate exam rooms where they each played the role of a patient presenting to the ED with chest pain.
After a briefing in a nearby classroom during which the instructor explained to the students the objectives and format of the activity, the instructor presented the first two slides of the labs and imaging PowerPoint presentation (Appendix E) on a large display monitor for students to review. These two slides contained the patient's chief complaint, vital signs, and electrocardiogram (ECG). Students were then divided into two groups of four to five, with each group led into one of the two exam rooms to collectively conduct a history and perform a physical exam on their respective SP. After 15 minutes in the exam rooms, all students reconvened in the classroom where the instructor facilitated a group discussion about how this simulated patient would hypothetically be managed in the ED setting, as described in the facilitator guide (Appendix A). Simulated diagnostic results were reviewed as a group by displaying the rest of the PowerPoint presentation (Appendix E), and the hypothetical case ended with the patient being administered aspirin and admitted to the observation unit. The initial briefing, SP encounter, and ensuing group discussion lasted 60 minutes total.
After the SP activity, each student individually wrote a patient note based on the hypothetical case and outcome presented in the SP encounter and discussion. Students were provided with copies of the ED record (Appendix F) and case ECG (Appendix G) to use as reference. They were also given the note template (Appendix H) to write their practice note in. Each student was required to submit this note assignment electronically before the deadline of 11:59 pm on the day of the activity. Each note assignment was reviewed by the instructor, who provided each student with one to two paragraphs of individualized written feedback on the quality of their practice note within 2 days of submission.
The final component of this curriculum was the provision of timely feedback on students’ first real patient notes. At the end of each student's first ED clinical shift, the supervising emergency physician reviewed all of the student's patient notes from that shift and provided verbal and written feedback on the student's documentation and overall performance using the shift evaluation rubric (Appendix I). The criteria listed in the evaluation rubric were modeled after the AAMC's fifth Entrustable Professional Activity regarding documenting a clinical encounter,15 which was used as a framework for assessing documentation proficiency.
The shift evaluation rubric also served as a summative assessment. If the supervising physician attested within the evaluation rubric that a student's notes from the first shift were both accurate without concerning errors and ready to serve as the primary record of the ED encounter within the medical record, the student was then entrusted with primary documentation responsibility for all ED patients in whose care they participated significantly throughout the rest of the clerkship. Student notes were composed within an institutionally standardized, prepopulated EHR template and were immediately viewable within a patient's medical record once electronically signed by the student. Faculty then revised and cosigned these patient notes according to institutional policy and in line with CMS guidance regarding the use of student notes for billing purposes.
To measure the effectiveness of our curriculum from the student perspective, we presented each student with an initial survey (Appendix J) to complete anonymously on the first day of the clerkship prior to delivering any documentation instruction. The initial survey also included a needs assessment regarding any previous documentation training students may have already had. We compared preintervention student responses to postintervention responses collected through a follow-up survey (Appendix K) completed anonymously by students during their mid-rotation feedback session that took place 2 weeks after the start of the clerkship. Preintervention and postintervention responses were compared using a two-tailed t test.
To more robustly measure the effectiveness of our training, we also conducted a cross-sectional survey of EM faculty physicians within our academic ED regarding medical students’ documentation skills after participating in our curriculum. We created a seven-question web-based survey (Appendix L) with response choices modeled after Likert-type scale response anchors.16 Survey questions assessed faculty perception of the quality and utility of medical student notes, as well as any change in the amount of face-to-face interaction faculty had with students and students’ utility on the care team when students wrote notes. The survey was distributed by email to all 36 full-time EM faculty physicians within our academic ED over a 2-month period starting 2 months after initial implementation of the new curriculum. All faculty responses were recorded anonymously to prioritize honesty.
A total of 105 medical students (36 third-years and 69 fourth-years) completed the documentation curriculum. All sessions were facilitated by the primary author, an EM physician and medical education fellow.
Out of 88 medical student respondents, 30% had received no formal training and 36% had received less than 30 minutes of training on writing a high-quality ED patient note prior to participating in our curriculum (16%: 30–60 minutes, 14%: 1–2 hours, 5%: greater than 2 hours). Fifty-six percent of students had received no formal training and 30% had received less than 30 minutes of training on how the contents of an ED patient note affects billing and coding (9%: 30–60 minutes, 3%: 1–2 hours, 2%: greater than 2 hours).
Medical Student Self-Assessment
We received 70 responses (67% response rate) to the preintervention initial survey and 66 responses (63% response rate) to the postintervention follow-up survey. The mean preintervention score for student comfort with writing a high-quality note was 4.1 on a 7-point scale (1 = extremely uncomfortable, 7 = extremely comfortable). After completing the curriculum, the mean score increased to 5.9 (p < .001). The mean preintervention score for student knowledge about billing and coding was 2.9 on the same 7-point scale, which increased to 5.5 after the intervention (p < .001; Table 1).
Faculty Physician Assessment
Out of 46 students for whom completed shift evaluation rubrics from their first shift in the ED were available for review, 96% were assessed by their supervising physician as having produced accurate documentation without concerning errors and being ready to have their documentation serve as the primary note in the medical record. Four percent did not meet one or both of these criteria and underwent individualized remedial instruction by a faculty physician prior to being entrusted with primary documentation for ED patients.
Faculty Physician Survey
We received 29 responses (81% response rate) to our faculty survey, which was deployed after implementation of the student curriculum. When asked how often medical student notes were useful for clinical documentation, 10% of surveyed faculty reported that they were always useful, 69% stated they were usually useful (defined as 90% or more of the time), and 14% reported they were frequently useful (defined as about 70% of the time); the remainder selected occasionally, and none selected sometimes, rarely, or never (Table 2). Three percent of faculty felt that medical student notes always contained enough information for billing and coding, with 38% reporting that they usually did (defined as 90% or more of the time) and 45% stating that they frequently did (defined as about 70% of the time); the remainder selected sometimes, and none selected occasionally, rarely, or never (Table 2). Ninety-four percent of faculty strongly agreed or agreed that overall, it was worth reviewing and cosigning medical student notes compared to writing the notes themselves (66% strongly agreed, 28% agreed, 3% neither agreed nor disagreed, 3% disagreed, and 0% strongly disagreed; Table 3).
Sixty-eight percent of faculty reported having more face-to-face interaction with students on shift when students wrote notes compared to when they did not (34%: much more interaction, 34%: somewhat more interaction, 28%: neither more nor less interaction, 3%: somewhat less interaction, and 0%: much less interaction). Ninety-seven percent of faculty found medical students to be a more useful part of the team when students wrote notes compared to when they did not (69%: much more useful, 28%: somewhat more useful, 3%: neither more nor less useful, 0%: somewhat less useful, and 0%: much less useful).
When asked what about the curriculum was working well, representative free-text faculty responses included the following:
“Overall level of note quality is excellent.”
“The student notes are awesome! They are often better than many resident notes and provide a great educational opportunity.”
“[Students] feel engaged, valued. Deeper and more thoughtful medical decision-making.”
“The training that they get before starting must be good, because they all seem to know what to do in advance.”
When asked what about the curriculum could be improved, representative responses included “More instruction to limit how verbose some of these notes can be” and “[Students] do miss some of the things that come with experience like not recognizing that some of the ‘pre-populated’ fields are actually empty.”
Overall, the documentation curriculum that we developed appeared to be effective at training third- and fourth-year medical students on proficient patient care documentation in EM. As revealed by our needs assessment, third- and fourth-year medical students often received little to no formal education on patient care documentation, which could potentially hinder their successful transition into postgraduate residency training. Our curriculum aimed to bridge this knowledge and skills gap by utilizing established principles of experiential learning and adult learning theory. After completing our orientation day curriculum, students were more confident in writing a high-quality note and were more knowledgeable about the components of a complete patient note as measured by their preintervention and postintervention surveys. The vast majority of students were also judged by their supervising physician to be able to document accurately and competently during their first shift in the ED. This finding was further corroborated by the faculty physician survey, which found that attending emergency physicians judged medical students as a whole to be proficient at writing high-quality, clinically useful patient notes after implementation of our curriculum.
One notable benefit of this curriculum is the relatively short amount of synchronous learning time required of students and instructors: 90 additional minutes at the very beginning of the clerkship. This condensed time investment could make it easier to integrate the curriculum within a medical student clerkship orientation setting among other competing educational objectives. Furthermore, the SP activity's secondary objective of introducing students to the emergency differential diagnosis and workup of chest pain allowed it to serve a dual purpose within the same time constraints. Another factor that likely aided in the success of this curriculum was the allowance of medical student notes to count for billing throughout the rest of the clerkship after successful completion of the curriculum. This prominent role of the student note in the care of real ED patients could have further motivated students to excel at this skill, as well as facilitating faculty buy-in for providing thoughtful feedback to students about their documentation.
Initially, there was some concern that the responsibility of patient care documentation would detract from the medical student experience during the clerkship, but our survey results appear to contradict this. In fact, the majority of faculty physicians found themselves having more face-to-face interaction with medical students while on shift when students were responsible for writing patient notes. A likely explanation for this finding is that the medical students were better integrated into the care team when they wrote patient notes, allowing for more robust discussions about patients and care plans and facilitating more opportunities for real-time feedback between faculty and students. This explanation is further corroborated by the fact that nearly all faculty survey respondents found students to be a more useful part of the team after completing our documentation curriculum.
One limitation of our curriculum is that the evaluation tool used to judge its success placed heavy emphasis on students’ self-assessment of their acquisition of knowledge as well as on faculty physician perception of students’ documentation proficiency as a group. Both of these measures are potentially vulnerable to subjective interpretation and variability among respondents. Future studies could explore assessing student documentation performance in an even more objective and standardized fashion, such as by having student notes examined by professional medical coders. One potential barrier to widespread implementation of our curriculum is that not all institutions allow medical student notes to count for billing purposes. Although this documentation curriculum can still be successfully implemented under those conditions, the fidelity of the student experience compared to real-world practice and the incentive for supervising physicians to provide valuable feedback on student notes could both be diminished if student notes cannot count for billing.
When we implemented this curriculum at our institution, one lesson we learned was the importance of garnering faculty buy-in and support for the project during the initial stages of development. Although we delivered and led the learning activities, we also relied on the broader faculty physician group to provide meaningful end-of-shift documentation feedback to our students and to entrust them with primary documentation responsibilities for ED patients. The impetus for developing this curriculum was to enhance medical student engagement during the EM clerkship, which allowed for effective messaging to faculty that this curriculum was a genuinely learner-centered educational initiative. One significant challenge we encountered in implementation was ensuring that student-written notes were compliant with institutional guidelines regarding teaching physician involvement and attestation. This required careful coordination with the billing and compliance teams within our department and health system. Previous advocacy for the value of student-written notes by our medical school leadership helped ease this process.
Future work in documentation education for medical students could involve adapting this curriculum to an asynchronous remote-learning format, which would allow for greater flexibility in implementation. The format of the curriculum could also be expanded to other medical school clerkships to provide similar benefits of increased documentation proficiency across other medical specialties. Since many of the core principles of effective documentation covered in the curriculum are shared among other clinical settings, an adapted version for another specialty could focus on specialty-specific documentation best practices for higher yield. Both of these directions for future efforts are currently being explored at our institution. Finally, more long-term longitudinal reassessments of students’ documentation proficiency could be studied to evaluate knowledge retention after completing the initial curriculum.
Hammoud MM, Margo K, Christner JG, Fisher J, Fischer SH, Pangaro LN. Opportunities and challenges in integrating electronic health records into undergraduate medical education: a national survey of clerkship directors. Teach Learn Med. 2012;24(3):219–224. https://doi.org/10.1080/10401334.2012.692267Google Scholar,
Hammoud MM, Dalrymple JL, Christner JG, et al. Medical student documentation in electronic health records: a collaborative statement from the Alliance for Clinical Education. Teach Learn Med. 2012;24(3):257–266. https://doi.org/10.1080/10401334.2012.692284Google Scholar,
Wittels K, Wallenstein J, Patwari R, Patel S. Medical student documentation in the electronic medical record: patterns of use and barriers. West J Emerg Med. 2017;18(1):133–136. https://doi.org/10.5811/westjem.2016.10.31294Google Scholar,
Obeso V, Brown D, Aiyer M, et al., eds. Toolkits for the 13 Core Entrustable Professional Activities for Entering Residency. Association of American Medical Colleges; 2017.Google Scholar
- 5. E/M service documentation provided by students (manual update). Centers for Medicare and Medicaid Services Medical Learning Network. February 5, 2018. Updated June 1, 2018. Accessed July 14, 2020. https://www.cms.gov/Outreach-and-Education/Medicare-Learning-Network-MLN/MLNMattersArticles/Downloads/MM10412.pdfGoogle Scholar
Hoonpongsimanont W, Velarde I, Gilani C, Louthan M, Lotfipour S. Assessing medical student documentation using simulated charts in emergency medicine. BMC Med Educ. 2018;18:203. https://doi.org/10.1186/s12909-018-1314-zGoogle Scholar,
Stice JE. Using Kolb's learning cycle to improve student learning. Eng Educ. 1987;77(5):291–296.Google Scholar
Guth T, Morrissey T. Documentation of EM encounters. Clerkship Directors in Emergency Medicine. Updated 2015. Accessed July 17, 2021. https://www.saem.org/about-saem/academies-interest-groups-affiliates2/cdem/for-students/online-education/m3-curriculum/documentation/documentation-of-em-encountersGoogle Scholar
Patel A, Ali A, Lutfi F, Nwosu-Iheme A, Markham MJ. An interactive multimodality curriculum teaching medicine residents about oncologic documentation and billing. MedEdPORTAL. 2018;14:10746. https://doi.org/10.15766/mep_2374-8265.10746Google Scholar,
Sarzynski E, Foley K, Riekse R, et al. An OSCE to assess trainee management of complex older adults and to teach documentation, coding, and billing. MedEdPORTAL. 2014;10:9903. https://doi.org/10.15766/mep_2374-8265.9903Google Scholar
Talwalkar J, Ouellette J. A structured workshop to improve chart documentation among housestaff. MedEdPORTAL. 2009;5:5095. https://doi.org/10.15766/mep_2374-8265.5095Google Scholar
Hirsh E, Harrison R. Billing, coding, and documentation (slightly unorthodox, but accurate). MedEdPORTAL. 2010;6:7742. https://doi.org/10.15766/mep_2374-8265.7742Google Scholar
Zavodnick J, Kouvatsos T. Electronic health record skills workshop for medical students. MedEdPORTAL. 2019;15:10849. https://doi.org/10.15766/mep_2374-8265.10849Google Scholar,
Shen E, Cristiano JA, Ellis LR. The electronic health record objective structured clinical examination station: assessing student competency in patient notes and patient interaction. MedEdPORTAL. 2020;16:10998. https://doi.org/10.15766/mep_2374-8265.10998Google Scholar,
Obeso V, Brown D, Phillipi C, et al., eds. Core Entrustable Professional Activities for Entering Residency—EPA 5 Toolkit: Document a Clinical Encounter in the Patient Record. Association of American Medical Colleges; 2017.Google Scholar
Vagias WM. Likert-Type Scale Response Anchors. Clemson University; 2006.Google Scholar