Original Publication
Open Access

A Structured Framework and Resources to Use to Get Your Medical Education Work Published

Published: January 17, 2018 | 10.15766/mep_2374-8265.10669

Appendices

  • Workshop Agenda.docx
  • Crossing the Finish Line.pptx
  • Introduction Example 1.docx
  • Introduction Example 2.docx
  • Worksheet Deliberate Practice.docx
  • Worksheet Self-Regulated Learning Theory.docx
  • Additional Resources.docx
  • Checklist for Authors.docx
  • Workshop Evaluation.docx

All appendices are peer reviewed as integral parts of the Original Publication.

To view all publication components, extract (i.e., unzip) them from the downloaded .zip file.


Abstract

Introduction: Medical educators often have great ideas for medical education scholarship but have difficulty converting their educational abstract or project into a published manuscript. Methods: During this workshop, participants addressed common challenges in developing an educational manuscript. In small-group case scenarios, participants discovered the importance of the “So what?” in making the case for their project. Incorporating conceptual frameworks, participants chose appropriate outcome metrics, discussed how to frame the discussion section, and ensured appropriate journal fit. After each small-group exercise, large-group discussions allowed the small groups to report back so that facilitators could highlight and reinforce key learning points. At the conclusion of the workshop, participants left with a checklist for creating an educational manuscript and an additional resources document to assist them in avoiding common pitfalls when turning their educational abstract/project into a publishable manuscript. Results: This workshop was presented in 2016 and 2017. Presenter evaluations were completed by 33 participants; 11 completed conference evaluations. The mean overall rating on presenter evaluations was 4.55 out of 5, while the conference evaluations mean was 3.73 out of 4. Comments provided on both evaluation tools highlighted the perceived effectiveness of the delivery and content. More than 50% of respondents stated that they planned to incorporate the use of conceptual frameworks in future work. Discussion: This workshop helped participants address common challenges by providing opportunities for hands-on practice as well as tips and resources for use when submitting a medical education manuscript for publication.


Educational Objectives

By the end of this activity, participants will be able to:

  1. Examine how to incorporate conceptual frameworks into medical education research.
  2. Assess how to incorporate higher-level outcome metrics to demonstrate the impact of their medical education project.
  3. Explore the top challenges faced when writing an educational manuscript.

Figures and Tables

  Table. External Evaluation Results (2016 and 2017)
Item
M (SD)a
Overall rating
3.73 (0.47)
Information relevant/useful
3.91 (0.30)
Met objectives
3.73 (0.47)
Allowed appropriate time for discussion
3.73 (0.47)
Provided practical information and skills training with learners as active participants
3.73 (0.47)
Expertise of leaders
3.82 (0.41)
Use of session time
3.82 (0.47)
  aUsing a 4-point scale where 4 = Outstanding.

Introduction

In many academic institutions, clinician educators face increasing expectations to produce scholarship and educational research.1 The absence of scholarly activity leads to slower academic advancement.2 Promotion committee chairs may place even more emphasis on publications (and external grant support) for clinician educators than department chairs do.3 Medical education faculty often have difficulty converting their educational project or abstract into a published manuscript.4 A study looking at publication rates of abstracts presented at the Research in Medical Education Conference and Canadian Conference on Medical Education found that only 34.7% (156 out of 449) of medical education abstracts were subsequently published within 6 years of presentation.5 Likewise, higher-quality studies, as measured by the Medical Education Research Study Quality Instrument, were associated with editor decisions to send manuscripts for peer review, invite revisions after review, and ultimately accept for publication.6 Therefore, it is critical that academic educators be equipped with the necessary skills to turn their innovative ideas into scholarship that contributes to advancing the field.7,8

While previous publications in MedEdPORTAL provide guidelines to conduct medical education scholarship,9,10 they do not incorporate the use of conceptual frameworks or tips and tools to overcome the top challenges faced when trying to get an educational manuscript published. Hodges provides key foundational principles to consider in designing professional development programming to enhance research skills,11 as well as a primer published in 2011 outlining the critical steps to transform an idea into a research question and ultimately a publishable project.7 The primer, a guide for faculty new to medical education research, is a thorough introduction but does not provide hands-on activities to facilitate learning through the application of concepts to actual educational projects. Our workshop activities walk participants through the steps in using a conceptual framework as a foundation for project design and for interpretation and analysis of results.

Exercises used in this session highlight rigorous methodologies for research design and assessment of high-level outcomes, key elements necessary for publication. The workshop also highlights effective writing techniques and journal fit as important considerations for prospective authors. The target audience for the workshop includes residents, fellows, and faculty interested in medical education who have some prior experience with medical education studies and/or scholarly work in education.

Methods

This 3-hour workshop was selected through a peer-reviewed process to be presented at the annual conference of the Pediatric Academic Societies in 2016 and 2017. The session could be presented in departmental or institutional settings in addition to other multi-institutional forums. All facilitators had served as editors for an academic journal. In general, facilitators should have knowledge of, and experience in, taking a medical education project from an idea through implementation to submission and publication as scholarship. Participants would benefit from prior knowledge of the definition of educational scholarship and from previous experience in presenting medical educational innovation/scholarly work in local, regional, or national forums, although information about these topics is included in the introduction to the workshop.

To facilitate the various instructional strategies used in the workshop, room setup should include a screen, LCD projector, laptop with internet connection (if available, but not necessary), round tables with six to eight participants per table, and a flip chart with markers at the front of the room.

The workshop agenda (Appendix A) described the workshop activities, which were mapped to learning objectives and a corresponding time line. The workshop began with introduction of speakers (Appendix B), followed by a needs assessment of the audience using an audience response system. This needs assessment could also be accomplished with audience members raising their hands or a facilitated large-group discussion, as well as a flip chart to record responses.

After the needs assessment, we described the workshop objectives and, as a road map for the session, provided an agenda that began with challenges encountered in publishing and how to overcome them and ended with practical tips for educational scholars. The challenges were presented in the order typically encountered during the design and implementation of research and were connected to the section of a manuscript where these elements were addressed. The first challenge we discussed was establishing the “So what?” for educational work, both to ensure that the research question was important and to convince the audience (editors and, ultimately, readers) that the question was important (Appendix B). The first small-group exercise allowed the participants to apply what they had learned about establishing significance by reading a draft introduction (Appendix C) and determining whether it built a convincing case for why the problem was important, whether it illuminated a gap in the literature, whether it addressed how the study would fill that gap, and whether it clearly stated the study’s specific aim/hypothesis. The exercise was facilitated by workshop leaders at each table (15 minutes) and was followed by a large-group report-out and debrief (5 minutes). We then supplied the participants with the published version of the introduction (Appendix D) and asked them to consider how the authors had addressed these issues in the final format.

The second challenge we discussed was identifying a conceptual framework in, and applying it to, educational research. Our second exercise allowed the participants to apply what they had learned about conceptual frameworks by developing a research question for a medical education topic through the lens of one of two different conceptual frameworks, deliberate practice (Appendix E) or self-regulated learning (Appendix F). Half of the group was assigned deliberate practice (Appendix E), and the other half was assigned self-regulated learning (Appendix F). We used the topic of teaching residents to initiate a management plan for a patient who was clinically decompensating; however, a variety of other topics could have been used. The small-group exercise was facilitated by workshop leaders at each table (15 minutes) and followed by a large-group report-out of the research question developed for each conceptual framework. The large-group report-out allowed a facilitated discussion comparing and contrasting how the different conceptual frameworks resulted in the development of different research questions (10 minutes). We then took a 10-minute break.

The third challenge we discussed was methodological flaws (Appendix B). Our third exercise allowed the participants to apply what they had learned about the selection of methodology by outlining an appropriate study design for the research question they had posed using their assigned conceptual framework in the previous exercise. The participants were asked not only to consider study design (quantitative/qualitative approach and specific methodology) but also to identify the study population, intervention to be studied, study instrument/tool, measurable outcomes, and analysis of results. The small-group exercise was facilitated by workshop leaders at each table (20 minutes). We concluded with a large-group report-out (15 minutes) to discuss how participants would assure methodological rigor in their study design. We then took another 10-minute break.

The fourth challenge we discussed was how to frame a paper’s discussion section in order to avoid common writing errors (Appendix B).

We concluded the workshop with tips on determining journal fit and lessons we had learned when conducting and publishing our own educational scholarship. Time was allotted for participants to ask questions (15 minutes). We provided the participants with a list of additional resources (Appendix G), which contained examples of common conceptual frameworks and tips to avoid common writing mistakes. We also gave them a checklist for use before they submitted their educational scholarship for publication (Appendix H).

The workshop was evaluated in two ways. Given the challenge of low response rates on conference evaluations sent by conference organizers, we developed an internal presenter survey to obtain additional data, including qualitative feedback, that we could use to improve the workshop for future iterations (Appendix I). This internal presenter evaluation was distributed at the end of the workshop and was completed anonymously (participants left completed evaluations on the tables in the rooms). Participants were asked to rate the overall quality of the workshop on a 5-point scale (1 = poor, 5 = outstanding). Participants were also asked a series of three open-ended questions:

  • “What was particularly effective?”
  • “What was least effective/how could we improve this workshop?”
  • “What is the one thing you learned today that you plan to incorporate in your future scholarship?”

In addition, the workshop was evaluated on a 4-point scale with an electronic standardized workshop evaluation used for all Pediatric Academic Societies conference workshops. This conference evaluation provided space for open-ended comments from participants.

Results

Learners were pediatric faculty and fellows-in-training from various disciplines who attended the 3-hour workshop at the Pediatric Academic Societies national conference in 2016 and 2017. Each year, approximately 35 learners attended the workshop. A total of 33 participants completed internal evaluations (17 in 2016 and 16 in 2017); comments were provided by all respondents. Internal evaluation rated the overall quality, on a 5-point scale (5 = outstanding), as 4.55 (SD = 0.56).

Eleven learners (eight in 2016 and three in 2017) completed external evaluations as well (Table), and 10 respondents provided comments.

  Table. External Evaluation Results (2016 and 2017)
Item
M (SD)a
Overall rating
3.73 (0.47)
Information relevant/useful
3.91 (0.30)
Met objectives
3.73 (0.47)
Allowed appropriate time for discussion
3.73 (0.47)
Provided practical information and skills training with learners as active participants
3.73 (0.47)
Expertise of leaders
3.82 (0.41)
Use of session time
3.82 (0.47)
  aUsing a 4-point scale where 4 = Outstanding.

Comments to the prompts on the internal evaluations and the open-ended comments on the external evaluations were analyzed together by three of the workshop presenters. Participants commented on the following strengths:

  • Delivery of workshop: varied perspectives of session leaders (facilitators rotating to different tables during exercises), opportunity for discussion, integrated hands-on activities, and examples provided to illuminate concepts.
  • Content of the workshop: practical tips and tools/resources provided.

Participants also offered suggestions for improvement. Some would have liked smaller groups, while others expressed that when table groups were split, they did not have the benefit of having as many perspectives represented in the discussion. Many commented that more time could have been spent on different components of the workshop, such as where to publish. Some suggested that additional exercises could be included, such as critiquing the methods section and/or discussion section of a manuscript or reviewing a successful example of a manuscript that used a conceptual framework.

In response to the prompt on the internal evaluation asking participants to state one thing they had learned in the workshop that they would incorporate in future scholarship, more than half identified the use of conceptual frameworks. Many participants noted the benefit of collaboration and expanding one’s expertise through working with others outside of medicine or with those who have published educational work in the past. Participants also noted the importance of a systematic planning process and stated that they planned to use the checklist as a guide on their next project.

Discussion

This workshop added to existing guides to help clinician educators engage in educational research by providing tools and practices that let participants gain experience in addressing common challenges submitting a medical education manuscript for publication. The session tackled critical considerations in designing, implementing, and publishing educational research, including applying a conceptual framework, making a case for the importance of the work, and utilizing rigorous methods to measure and demonstrate higher-level outcomes.

Participants appreciated both the content and delivery of workshop materials, particularly the opportunity to hear from workshop leaders experienced in publishing medical education scholarship and hands-on learning. The idea of conceptual frameworks was difficult for many workshop participants to grasp, so examples and hands-on application were felt to be particularly helpful. Participants may have found conceptual frameworks both valuable and difficult because even published studies in medical education do not always state the framework underlying the work. Due to this difficult concept, participants appreciated the balance between small-group work (with some expressing a desire for even smaller groups or individual mentorship) and large-group discussions, where multiple perspectives of workshop leaders and participants could be shared.

Workshop materials could be adapted to present individual sections as a series of 1-hour or 1.5-hour weekly or monthly workshops for fellows or junior faculty from a single institution. Breaking the workshop into smaller segments could reduce participant cognitive load and fatigue. Smaller facilitator-to-participant ratios would maximize the benefit to participants by allowing facilitators to provide more focused attention to each participant. Ideally, one facilitator to three participants would allow facilitators to help participants through some of the more challenging small-group activities. Facilitators could remain with each small group rather than floating between tables yet still provide various perspectives during the large-group report-out/discussions. Participants could work on their own project as part of the workshop. In order to better facilitate this, participants could share their work with facilitators in advance, and the number of facilitators could be increased so that they would be able to work individually (1:1) or in pairs (1:2) with participants. We found that the use of conceptual frameworks was both the most challenging and the most rewarding part of the workshop for participants. In choosing conceptual framework examples for the exercises used in the workshop, facilitators could select simpler examples that audience members may have previously encountered to make it easier for them to understand and utilize conceptual frameworks during the workshop. Selecting topics that educators can easily relate to or educational hot topics, such as improving procedural competency or delivering bad news, would allow participants to see the practical application and relevance of concepts introduced in the workshop. If a topic is not provided to the participants up front, we suggest that the small groups could choose a topic that is timely and important to the learners in the session. Facilitators should be prepared to be flexible during the session in order to address the needs of participants who may have different levels of experience in educational scholarship.

We had several limitations. Our evaluations captured only short-term outcomes of learner perceptions and anticipated future use of workshop concepts. We did not receive evaluations from all participants. Evaluation tools were created by us (internal) and the meeting organizers (external) and have not been formally tested for validity and reliability. Facilitators and participants were all academic pediatricians, although we believe the content is relevant for clinician educators from various clinical disciplines. All facilitators were Academic Pediatrics journal editors, and it is possible that facilitators with less experience in medical education scholarship publication may have had more difficulty facilitating more nuanced or specific questions related to successful publication from participants. Future evaluation of this work could include gathering higher, longer-term outcomes, including whether participants incorporate what they learn into future medical education scholarship and whether participants are able to successfully publish their work.


Author Information

  • Su-Ting T. Li, MD: Professor, Department of Pediatrics, University of California, Davis, School of Medicine
  • Maryellen E. Gusic, MD: Professor, Department of Medical Education, University of Virginia School of Medicine
  • Robert J. Vinci, MD: Professor, Department of Pediatrics, Boston University School of Medicine
  • Peter G. Szilagyi, MD: Professor, Department of Pediatrics, University of California, Los Angeles David Geffen School of Medicine
  • Melissa D. Klein, MD: Associate Professor, Department of Pediatrics, University of Cincinnati College of Medicine

Disclosures
None to report.

Funding/Support
None to report.

Ethical Approval
Reported as not applicable.


References

  1. Roberts DH, Schwartzstein RM, Weinberger SE. Career development for the clinician–educator: optimizing impact and maximizing success. Ann Am Thorac Soc. 2014;11(2):254-259. https://doi.org/10.1513/AnnalsATS.201309-322OT
  2. Castiglioni A, Aagaard E, Spencer A, et al. Succeeding as a clinician educator: useful tips and resources. J Gen Intern Med. 2013;28(1):136-140. https://doi.org/10.1007/s11606-012-2156-8
  3. Atasoylu AA, Wright SM, Beasley BW, et al. Promotion criteria for clinician-educators. J Gen Intern Med. 2003;18(9):711-716. https://doi.org/10.1046/j.1525-1497.2003.10425.x
  4. Brocato JJ, Mavis B. The research productivity of faculty in family medicine departments at U.S. medical schools: a national study. Acad Med. 2005;80(3):244-252. https://doi.org/10.1097/00001888-200503000-00008
  5. Walsh CM, Fung M, Ginsburg S. Publication of results of abstracts presented at medical education conferences. JAMA. 2013;310(21):2307-2309. https://doi.org/10.1001/jama.2013.281671
  6. Reed DA, Beckman TJ, Wright SM, Levine RB, Kern DE, Cook DA. Predictive validity evidence for Medical Education Research Study Quality Instrument scores: quality of submissions to JGIM’s medical education special issue. J Gen Intern Med. 2008;23(7):903-907. https://doi.org/10.1007/s11606-008-0664-3
  7. Ringsted C, Hodges B, Scherpbier A. “The research compass”: an introduction to research in medical education: AMEE Guide No. 56. Med Teach. 2011;33(9):695-709. https://doi.org/10.3109/0142159X.2011.595436
  8. Chan TM, Thoma B, Hall AK, et al. CAEP 2016 Academic Symposium: a writer’s guide to key steps in producing quality medical education scholarship. CJEM. 2017;19(S1):S9-S15. https://doi.org/10.1017/cem.2017.30
  9. Walvoord E, Hobson-Rohrer W, Bogdewic S. Getting promoted: turning your clinical work into scholarship. MedEdPORTAL. 2014;10:9694. https://doi.org/10.15766/mep_2374-8265.9694
  10. Tewksbury L, English R, Christy C, et al. Scholarship of application: when service is scholarship - a workshop for medical educators. MedEdPORTAL. 2009;5:7734. https://doi.org/10.15766/mep_2374-8265.7734
  11. Hodges B. Faculty development for research capacity building. In: Steinert Y, ed. Faculty Development in the Health Professions: A Focus on Research and Practice. Dordrecht, the Netherlands: Springer; 2014:79-96.


Citation

Li S-TT, Gusic ME, Vinci RJ, Szilagyi PG, Klein MD. A structured framework and resources to use to get your medical education work published. MedEdPORTAL. 2018;14:10669. https://doi.org/10.15766/mep_2374-8265.10669

Received: September 23, 2017

Accepted: January 2, 2018