Veteran Interactive Cases to Reinforce Service (VICTORS): Standardized Patient Assessment of Veteran-Centered Care

Educational Objectives

By the end of this session, learners will be able to:

1. Practice and receive feedback on interpersonal communication skills in the clinical setting.

2. Improve skills related to addressing mental health issues that particularly affect U.S. military veterans and others.

3. Recognize knowledge deficits and learning needs through a reflective self-assessment process.

Introduction

The last decade of American history has been marked by extensive armed conflict, with deployment of hundreds of thousands of troops to combat zones in Iraq, Afghanistan, and other parts of the world. A substantial number of military personnel are now returning to civilian life with unique medical needs, and as they do, health professional educators must turn their attention to the development of patient-centered competency curricula regarding the medical care of U.S. military veterans.^1–4 Mental health conditions like post-traumatic stress disorder (PTSD) and traumatic brain injury carry their own morbidity and mortality. Veterans with mental illness are more likely to abuse alcohol and opioid medications, and recent studies indicate higher rates of suicide among returning veterans when compared to nonveterans.^5–7 Female veterans and veterans with PTSD from military sexual trauma (MST) are subject to unique mental health consequences as well.⁸ There are few experiential educational cases that focus on veteran-centered care. Currently, there are few clinical skill development tools that focus on veteran-centered care.

The Joining Forces initiative was recently launched to establish and improve the nation's commitment to meeting the unique health care needs of veterans and their families.⁹ In response to the volume of veterans returning from combat who are seeking care within and outside of the Veterans Health Administration (VHA) health care system, it is imperative that future physicians receive sufficient training to provide the care necessary to ensure veterans' optimal health.^2,10–12 We developed two distinct standardized patient (SP) cases geared toward improving medical trainees' skills and knowledge in assessment and management of patients with PTSD and MST.

The target audience for these case scenarios includes residents, fellows, and medical students and can be can be modified for clinical medical students in clinical rotations and for adult and psychiatry hospitalists. These cases can provide data for milestone assessment for specialties and improve veteran-centered patient communication skills. This feedback can help improve trainee comfort with interacting with veterans. In addition to communication skills and knowledge of alcohol and acetaminophen use, these cases will also assess knowledge of PTSD and MST and their treatment using a content checklist. The checklist includes the assessment of medical and social history, military history, and communication skills. These materials also include optional self-assessment and remediation activities.

In the SP case scenarios (Appendix A), case A features an Army veteran with PTSD presenting with insomnia and irritability, while case B features a Navy veteran with MST presenting with headaches and anxiety. These cases have been delivered as an unannounced SP to residents. We have used both cases with over 60 resident/SP initial encounters in the VHA health care system. The cases were also piloted in the SP center on a wide range of learners, including students, residents, and faculty. Both cases were used as an assessment.

There were three stages of development for this resource: case development, checklist development, and case and checklist review.

The case development team was comprised of experts in the fields of medical education, education assessment, psychiatry, and SP training. Clinicians with experience caring for patients with PTSD and MST in both VHA and civilian health care settings used their experiences to develop the case portrayal content. A member of the team captured those salient points, as well as reviewed the literature for common signs and symptoms of presentations, to ensure a representative case for PTSD and MST. The cases were developed to address potential deficits in learning and help trainees establish sound methods for translating these skills.

The checklist (Appendix B) was developed by our expert panel consisting of faculty from internal medicine and psychiatry, standardized patient educators (SPEs), and a medical education objective standardized clinical examination expert. The checklist used content from the Diagnostic and Statistical Manual of Mental Disorders,¹³ the VHA health system clinical alert/reminders and screening questions related to PTSD and MST, and checklist items from previous research studies to determine specific domains for assessment and ensure common presentations of PTSD and MST in the clinical setting.

We asked faculty members who were not involved with the project and who also had experience with SPs in the clinical setting to review the case and checklist materials. In this stage, we reviewed their input and made minor necessary changes to improve the validity of the presentation.

Methods

These cases were designed to be deployed in the clinical setting as announced or unannounced patients. The cases can be included in standard scenarios or cases at an SP center. If a facilitator is interested in the deployment in a clinical setting, the facilitator should cultivate relationships with training program directors and clinical management personnel to understand the logistics of adding a patient to a learners' clinic session. There are possible implications for the medical record and real-patient access that must be considered. Once the placement of the case is determined, there are several steps involved in the training of SPs to be able to adequately portray these cases.

SP Training

The SPs were trained and evaluated by SPEs. SPs also completed practice interviews with senior medical students, residents, and faculty prior to formally portraying the case. The training methods included the following:

At-home case preparation (2 hours): SPs receive the case scenario and a worksheet to complete 7 to 10 days prior to the first on-site training. During this stage of training, SPs focus on learning the role by reviewing the checklist and patient details. This stage ensures SPs arrive at the first training session prepared. Details are also provided covering PTSD and MST definitions and symptoms, military conflicts, military terminology, deployment screening questions, and reintegration challenges of recently returning veterans (e.g., substance abuse, emotional issues, social relations, etc.).

On-site training (4–6 hours): During this training, the SPs review and discuss the materials they were sent and focus on the best ways to interpret the scenarios, asking any questions about the case and checklist they may have and determining appropriate responses to questions likely to arise during the encounter.

At-home communication skills training (1 hour): SPs independently study the communication skills aspects of a case. SPs learn the 10 communication skills items on which they will need to score the trainees through videos, slides, and quizzes.

Practice interview (1 hour): This stage of the training focuses on applying the role via practice interview with an advanced-level student. SP performance is evaluated by SPEs with written feedback.

Postpractice interview training (1–2 hours): SPs review their practice interview video and receive the performance review to standardize role portrayal and scoring.

Posttraining survey (15 minutes): SPs also complete a posttraining survey.

Audio transcript review: Based on transcripts, SPEs complete a performance review of SP role portrayal and scoring to ensure case reliability.

Given that these have been used as formative exercises, we provide a sample patient-provider encounter for each case (Appendices C & D) that can be used for trainee remediation as well as for training portrayal purposes. To provide more context to the exercise, the remediation assignment (Appendix E) provides additional resources for trainees as a reference point for their activities and future encounters. Finally, we have mapped the course objectives and assessment to resident learner milestones (Appendix F).

Results

To establish content validity, we convened an expert panel to review the initial checklist. We asked 11 experts (faculty and clinical staff) to rate the five domains assessed by the checklist: Military History, Communication Skills, Assessment, Triage, and Professionalism. The experts rated the domains on their relevance as a measure of a learner's skills; their ratings ranged from 1 = definitely do not include this task to 4 = definitely do include this task.

A mean for each item was calculated for each of the five domains. Item mean scores ranged from 2.36 (the only item mean under 3.25) to 4.00 (nine items had this mean). The mean scores for each domain were Military History = 3.51, Communication Skills = 3.88, Assessment = 3.65, Triage = 3.50, and Professionalism = 3.73. These results indicate expert support for the validity of the instrument.

We also conducted a pilot test of the two cases and the checklist with seven medical students, seven residents, and four faculty members. The results of an analysis of variance found significant differences by expertise level (p =. 006); medical students (63% ± 13%) and residents (70% ± 11%) scored lower than faculty (88% ± 2%). The effect size (η² =. 49) also indicated a large effect for expertise.

Validity Evidence

The 36-item checklist was found to be reliable (coefficient α = .83) and valid. The 10-item Communication Skills subscale had an internal reliability of .58. To examine validity, scores were compared by house officer (HO) level, and while HO3s scored higher (92% ± 12%) than both HO2s (84% ± 12%) and HO1s (84% ± 13%), differences were not statistically significant. However, the effect sizes (both .62) indicate practical significance. HOs performed well on Communication Skills, with a mean and standard deviation of 86% ± 13% and a range of 60% to 100%. Scores were similar for both cases.

Prior to the first encounter, SPs were evaluated by 13 senior medical students, seven residents, and three faculty, for a total of 23 evaluations (see Table 1). SPEs reviewed videos of practice interviews and provided both written and verbal feedback on performance. SPs also provided data on their confidence portraying the role after receiving training (see Table 2).

Table 1. Rater SP Evaluations (N = 23)

Table 2. Standardized Patient Posttraining Self-Evaluations^a (N = 4)

Due to the subject matter, we recommend the training include a brief overview of the military (e.g., recent conflicts, military terminology). These cases can be announced or unannounced. They can be in the clinical setting. They were in the VHA's Computerized Patient Record System as test patients. Also, instructors can edit the demographics, geographic location, and clinical presentation according to their site and what is appropriate for their learners.

Below are several comments that represent themes from most of the residents in the postevaluation remediation exercise. The residents noted the need for an improvement in their military history–taking skills and acknowledged it as an important part of the social history. They also noted their need to develop a higher comfort level with asking specific and concise questions related to PTSD and MST.

• “Again, being less fearful in going into the specifics of a person's mental illness is the major thing gained from this exercise.”

• “I need to develop a better social history that incorporates military history better.”

• “My military history was unorganized and non-specific. I need to ask specific questions relating to traumatic events, including combat details (injuries, deaths) and noncombat (violence, unwanted contact). I also did not ask specific questions about medications tried or ask about a psychiatric history. I did not know the details about what kind of resources are available at the VA for help with PTSD.”

• “I found that I asked questions very superficially because I was uncomfortable talking about PTSD and MST.”

The results begin to demonstrate an area of growth for resident education.

Discussion

Throughout this process, we have found it useful to have a close relationship with the program director and clinical staff to help facilitate the clinic visit process from start to finish. Several things were done to ensure validity in the clinical setting. Making the SP materials resemble actual clinic documents is important. Having the SPs dress as closely as possible to blend into the waiting area and fit into the surroundings was also key to our success. Having this support also lends credibility to the assessment by the learner. Allowing the SPs to visit the VHA to become familiar with military lingo (e.g., using sir and ma'am) when conversing with professionals was another technique used in our training.

It helps to identify SPs who not only can play the role but also are comfortable ad-libbing in context, as they will encounter a great deal of uncertainty, such as providing details about their hometown. One final issue that we continue to struggle with is getting feedback to the residents in a timely manner so they can implement what they may have learned, given the novel approach and the need to want to vet our findings and ensure our scoring rubrics are reliable.

A significant number of military personnel are now reintegrating into civilian life with unique medical needs, and as they do, medical personnel need to be prepared to provide the best care possible to U.S. military veterans.¹⁴ The use of SPs provides the opportunity to develop and assess interpersonal communication in ways that direct patient care cannot, for instance, through feedback from the SP or the opportunity to work with a veteran SP for trainees in programs without a VHA affiliation. Medical trainees will benefit greatly from having the experience in a safe environment to develop these skills.