Original Publication
Open Access

A Structured, Debate-Style Cardiothoracic Surgery Journal Club for Trainee Acquisition and Application of Seminal Literature

Published: December 23, 2016 | 10.15766/mep_2374-8265.10521

Appendices

  • Instructor Blueprint.pdf
  • Collection of Key Landmark Articles in Cardiothoracic Surgery.pdf
  • Thoracic Surgery Debate Topics.pdf
  • Thoracic Surgery Debate Articles.pdf
  • Thoracic Surgery Debate Patient Scenarios.pdf
  • Thoracic Surgery Debate Multiple-Choice Examination With Key.pdf
  • Cardiac Surgery Debate Topics.pdf
  • Cardiac Surgery Debate Articles.pdf
  • Cardiac Surgery Debate Patient Scenarios.pdf
  • Cardiac Surgery Debate Multiple-Choice Examination With Key.pdf
  • Evidence-Based Medicine Multiple-Choice Examination With Key.pdf
  • Debate Coin Toss.pptx
  • Debate Score Sheet.pdf

All appendices are peer reviewed as integral parts of the Original Publication.

To view all publication components, extract (i.e., unzip) them from the downloaded .zip file.


Abstract

Introduction: The acquisition of specialty-specific seminal literature and its application to daily, clinical patient-care decisions are critical components of clinical training. This structured, debate-style cardiothoracic surgery journal club module engages trainees in discussion of realistic patient scenarios, incorporating an extensive body of literature identified as the best evidence for the practice of cardiothoracic surgery. Methods: We designed the structured, debate-style cardiothoracic surgery journal club and delivered it to University of Texas MD Anderson Cancer Center cardiothoracic surgery trainees. Overall assessment of knowledge acquisition consisted of both direct judging of debates by faculty facilitators and a year-end written test of trainee knowledge. Associated materials include guidelines and resources for faculty facilitators and trainees to prepare them for the journal club debate. Also included are cardiothoracic surgery patient cases, PowerPoint presentation slides, a debate score sheet, and multiple-choice knowledge tests with answer keys. Results: Our structured, debate-style cardiothoracic surgery journal club is an effective educational intervention for cardiothoracic surgical trainees to gain practice in applying specialty-specific, literature-based evidence to particular patient problems. Discussion: This resource may be used by course directors for surgery, for independent study by individuals planning to matriculate into surgical residencies, or as a review for those already in surgical training. Moreover, this curriculum can be delivered at other clinical training programs.


Educational Objectives

By the end of this session, learners will be able to:

  1. Recall current best literature-based evidence in cardiothoracic surgery.
  2. Apply literature-based evidence to relevant clinical decision-making.
  3. Identify supplemental or new literature to support clinical decision-making.
  4. Evaluate and defend clinical decision-making founded on literature-based evidence.

Introduction

The practice of evidence-based medicine (EBM) requires the integration of clinical expertise with the best available evidence in order to conscientiously and judiciously make decisions about the care of individual patients.1,2 However, barriers to practicing EBM include, among others, rapidly advancing technology and challenges related to understanding how to select and apply appropriate literature to everyday situations.1 In order for surgeons to practice EBM, it has been suggested that critical appraisal skills should be introduced to residents and fellows during training.3 Journal clubs serve as one of the most commonly employed means of imparting EBM skills to trainees; unfortunately, the structure and efficacy of such efforts tend to vary widely, and there is no standardized process of conducting an effective journal club.1,4 Perhaps even more important is that there are limited assessments of the acquisition of these skills and their integration into one’s own clinical practice.

Previous authors have attempted to describe best practices in conducting journal club programs, with a systematic review suggesting a number of strategies that contribute to journal club efficacy.4 Successful journal clubs have included regular and anticipated meetings, mandatory attendance, clear long- and short-term purpose, appropriate incentives, and availability of articles, with use of the internet for wider dissemination and data storage.1,5 However, despite attempts to describe journal club efficacy, only a minority of papers involved the use of objective outcome assessment tools.4 When assessments were conducted, most focused on participants’ attitudes and reading habits.6-9 Knowledge of seminal, specialty-specific articles and their application to clinical scenarios are of key importance in clinical training, but educational studies demonstrating the efficacy of a journal club in achieving these ends are lacking.

The need for such programming has further relevance in the setting of the ACGME Next Accreditation System, which focuses on the use of educational Milestones to assess resident competence.10 Specialty-specific Milestones within the six ACGME core competencies are being used to measure program effectiveness. Thoracic surgery’s Milestones were presented by the Thoracic Surgery Directors Association at the 2013 meetings of the Society for Thoracic Surgery and the American Association of Thoracic Surgeons.11 While Milestones related to medical knowledge, patient care/technical skills, professionalism, and interpersonal/communication skills align easily with everyday training of cardiothoracic surgeons, it is not readily apparent how to best instruct and assess trainees in the realm of problem-based learning and improvement. However, early experience in other specialties has suggested that the journal club might prove to be an excellent tool for instruction, although it remains limited in its ability to assess these skills. Most clubs still use survey methods of self-elicited assessments or assess epidemiology skills with little or no assessment of applying EBM to clinical scenarios.12,13

To this end, we designed a structured, debate-style cardiothoracic surgery journal club aimed at addressing these needs for cardiothoracic surgical trainees at the University of Texas MD Anderson Cancer Center. We proposed a debate-style format, with monthly topics based on realistic patient scenarios, incorporating a vetted body of literature deemed to be of significance to the practice of cardiothoracic surgery. Though this curriculum has been studied with cardiothoracic surgical trainees, it could easily be adapted for use by a broader audience, including independent study by practicing surgeons, and can be delivered at other clinical training programs.

Methods

We introduced a structured, debate-style cardiothoracic surgery journal club (Appendix A) aimed at addressing key controversial topics in cardiothoracic surgery. Debate topics with accompanying clinical cases are carefully crafted to force participants into an area of existing debate within the cardiothoracic surgical community. Though elements of the case may have been resolved in the literature, the debate topics allow participants to utilize the data and best available evidence to support their position.

Journal club articles have been selected and debate topics crafted by a committee of academic cardiothoracic surgeons from two institutions involved with the study—University of Texas MD Anderson Cancer Center and University of Texas Health Science Center at Houston.

The curriculum is designed for cardiothoracic surgical trainees. The principles and skills of our structured, debate-style journal club can easily be adapted for use with learners in any specialty as well as for forms of continuing medical education.

The facilitators should be seasoned clinicians who have experience with basic EBM principles as well as knowledge of the debate topic to be discussed (thoracic surgery debate topics: Appendix C, cardiac surgery debate topics: Appendix G). The facilitators should have reviewed all materials before the session and should anticipate and be comfortable answering questions that arise during the session.

The journal club consists of 10 monthly sessions per academic year, with debate topics for thoracic and cardiac surgery. Note: There are three extra debate topics for cardiac surgery as a supplementary resource for facilitators to choose from. For each monthly session, two trainees are given a specific clinical scenario (thoracic surgery patient scenarios: Appendix E, cardiac surgery patient scenarios: Appendix I) with an accompanying position statement around which the debate is focused. Cases are drawn from scenarios frequently encountered in the clinical practice of cardiothoracic surgery. Each participant is assigned a pro or con position with regard to the position statement. Order of presentation is determined by a coin toss, and participants are given 20 minutes to present arguments to support their positions. After each side has completed its initial presentation, there is a 5-minute break to allow participants to make last-minute adjustments to their rebuttals. Participants are then each given 5 minutes to present their rebuttals, with the order of presentation reversed from the initial arguments. At the end of each session, voting determines a winner, who receives a small nominal award.

Our original schedule (Appendix L) was as follows:

  • 6:45 pm-6:50 pm (5 minutes): first affirmative.
  • 6:50 pm-6:55 pm (5 minutes): first negative.
  • 6:55 pm-7:00 pm (5 minutes): second affirmative.
  • 7:00 pm-7:05 pm (5 minutes): second negative.
  • 7:05 pm-7:25 pm (20 minutes): discussion period.
  • 7:25 pm-7:30 pm (5 minutes): break for preparation.
  • 7:30 pm-7:35 pm (5 minutes): negative summary/rebuttal.
  • 7:35 pm-7:40 pm (5 minutes): affirmative summary/rebuttal.
  • 7:40 pm-7:45 pm (5 minutes): voting.
  • 7:45 pm-8:00 pm (15 minutes): feedback.
  • 8:00 pm-8:05 pm (5 minutes): presentation of next debate topic.
  • 8:05 pm-8:07 pm (2 minutes): winner announced.

The use of a room of adequate size for the group with a computer, LCD projector, and screen will facilitate success for users of this module.

This structural, debate-style journal club format requires participants to be familiar not only with the literature supporting their own arguments but also with the literature of the opposing position. In addition, it allows participants to utilize critical appraisal skills in order to attempt to discredit the arguments of their opponent.

In preparation for the journal club session, once participants have taken the pretest, they must access the following:

  • Key and debate articles: an electronic collection of key seminal articles in cardiothoracic surgery deemed by a committee of academic cardiothoracic surgeons to have impacted the practice of the specialty of cardiothoracic surgery (Appendix B). Participants must also access preselected articles that are relevant to the debate topics (thoracic surgery: Appendix D, cardiac surgery: Appendix H). Of note, participants are expected to utilize this collection of articles but are not limited in their presentations to the articles in the collection.
  • List of patient scenarios: specific patient scenarios for each debate topic addressing controversial issues in cardiothoracic surgery for which adequate literature exists to support both sides and that supporting literature is considered to be among the key publications in the field.
  • Suggested EBM resources include:
    • What Is a Systematic Review?14
    • Checklist for appraisal of systematic reviews and meta-analyses.15
    • Other appraisal checklists (randomized controlled trials, cohort studies, etc.).16

We developed a scoring instrument to assess participants in the debate at each monthly session based on their identification of appropriate literature, their presentation of the data, and their appropriate application of the literature findings to the clinical scenario under debate. Participants are scored by faculty facilitators during monthly debates using the debate score sheet (Appendix M). Scoring categories include knowledge of subject, organization, use of supporting literature, persuasiveness, and overall effect.

Participants take a year-end written test to assess trainee knowledge acquisition. These multiple-choice examinations (thoracic surgery: Appendix F, cardiac surgery: Appendix J, EBM: Appendix K) consist of approximately two to three questions addressing each debate topic as well as eight questions regarding EBM levels of evidence.

The challenge with this structured journal club is getting through all the material in a 1.5-hour session. This is best accomplished via strict adherence to the suggested schedule.

If possible, an internet repository should be created where all articles and resources are kept (such as what we utilized: mdanderson.libguides.com/csjc). We also included clinical case descriptions for the debates, posting of completed debates, and a leader board on the website.

Results

We developed a module for a structured, debate-style cardiothoracic surgery journal club that aims to improve participant acquisition of specialty-specific, literature-based evidence and its application to particular patient problems. Evaluation of participants was based on the scoring instrument and participant performance on a multiple-choice examination. Cardiothoracic surgery trainee written exam results revealed a trend toward enhanced knowledge acquisition and retention as shown via improved scores on topics that were debated as compared to those that were not (9.8% vs. −4.2%, p = .105). Surveys completed by trainees (n = 6) and faculty (n = 11) with a response rate of 94.4% demonstrated that the debate-style journal club was deemed superior to the previous single-paper traditional journal club by encouraging greater participant engagement, enabling better application of literature to clinical scenarios, holding participants more accountable for the breadth of literature related to given topics, encouraging participants to explore the literature in greater depth, and requiring more critical evaluation of the cardiothoracic surgical literature.

Trainee ability to sway the position supported by the attendee in the debate was strongly correlated with trainee use of supporting literature (r = .853) and moderately correlated with persuasiveness (r = .465) and overall effect of the debate (r = .625). Organization of debate and knowledge of subject matter were weakly (r = .199) and not correlated (r = .027), respectively, with change in position supported by the attendee.

When participants were asked to grade their priorities for each journal club style, interestingly, the most important aspects of the traditional journal club became the least important aspects of the debate journal club. These aspects—including boredom, inadequate educational value for attendees and participants, breadth of papers covered being too broad or narrow, inadequate depth of topic covered, and inability of participants to apply discussion to real patients—were no longer issues of concern in the debate journal club. Whereas other concerns, such as food, location, and inability to know exactly which papers would be discussed, became concerns in the debate journal club.

In terms of favorite and least favorite aspects of the debate-style journal club, trainees admitted that preparing for a debate with a colleague required much more time and preparation than the traditional single-paper journal club format. As well, participants tended to dislike the winner and loser aspect of the debate and felt that success was less dependent on the strength of their arguments than on the position picked by the trainee, though these were balanced with the benefits of a more interesting and in-depth discussion of the literature in gaining a better understanding of the controversies and the evidence to support daily clinical decision-making. Additional objective feedback was gathered and consisted of laudatory remarks uniformly supportive of the innovative educational strategy.

Discussion

The acquisition of specialty-specific seminal literature and its application to daily clinical patient-care decisions are critical components of clinical training. In response to the new ACGME and the Next Accreditations Systems mandates requiring trainee competency in problem-based learning and improvement, we have designed a module to address this need.

A major weakness of the traditional single-paper journal club was that participants felt it was of inadequate educational value, given that discussions surrounded a too-narrow group of papers, and thus, they were uninterested and did not participate.17 The debate-style journal club has the advantage of providing an entertaining and informative debate on controversial topics as well as covering all the literature on a topic. Participants are required to be familiar not only with the literature supporting their own arguments but also with the literature of the opposing position. It is our hope that by taking a position supported by evidence and utilizing critical appraisal skills to discredit the arguments of their opponents, trainees will be able to provide a clear rationale for their practice and achieve improved responses on their board exams.

Incorporating the winner and loser aspect of competition to the journal club can improve engagement of participants, though some participants felt that their success was dependent on the side they were required to argue for. Despite these concerns, trainees still felt that grading was objective and fair, with scores received from multiple fronts, including their wins and losses, debate scores, feedback, and performance on pre- and postdebate examinations. This element of competition may be further enhanced by establishing a group identity to which the social comparison is extended, which can lead to increased group performance,18 though larger cohorts would be required for this comparison.

Our study is subject to several limitations. This pilot study is limited by its small data set and may have been too underpowered to show a significant difference in the analyses performed. This limitation is shared by many prior studies on journal clubs and educational outcomes. However, compared to prior studies that utilized self-assessment of performance, we believe we show important trends with objective assessment of improvement in problem-based learning skills and knowledge in cardiothoracic surgery. Despite our best efforts, not all faculty were able to attend all the sessions, and this may have reduced the sessions’ overall educational value. It is difficult to isolate the debate-style journal club from other learning encounters in problem-based learning that may have occurred independently throughout residency training.

We implemented this curriculum with a pilot group of trainees. Evaluation via multi-institutional expansion is needed to validate our preliminary findings with this trainee cohort. An aspect we hope to improve on in the future use of this curriculum is to attempt to use the same trainees as their own internal control by alternating between traditional journal club and debate styles. Assessment can then be done by the same scoring instrument, and participant performance on multiple-choice questions generated from topics discussed at both forms of the journal club can be compared. We believe these data would be invaluable to generate a control group with which to compare trainee performance and acquisition, as well as account for growth in trainee knowledge throughout the year.

Overall, this structured, debate-style cardiothoracic surgery journal club is an effective educational intervention for cardiothoracic surgical trainees to gain practice in applying specialty-specific literature-based evidence to particular patient problems. This resource may be used by course directors for surgery, for independent study by individuals planning to matriculate into surgical residencies, or as a review for those already in surgical training. Moreover, this curriculum can be delivered at other clinical training programs.


Author Information

  • Mara B. Antonoff, MD: Thoracic Surgeon, University of Texas MD Anderson Cancer Center
  • Tom C. Nguyen, MD: Cardiovascular Surgeon, University of Texas Health Sciences Center at Houston
  • Jessica G. Y. Luc: Medical Student, University of Alberta Faculty of Medicine and Dentistry
  • Clara S. Fowler, MSLS: Research Services and Assessment Manager, University of Texas MD Anderson Cancer Center
  • April Aultman Becker, MLIS: Dean of Libraries and Research Technologies, Sul Ross State University
  • Steven B. Eisenberg, MD: Cardiothoracic and Vascular Surgeon, McGovern Medical School at the University of Texas Health Science Center at Houston and the Memorial Hermann-Texas Medical Center
  • Randall K. Wolf, MD: Cardiothoracic and Vascular Surgeon, McGovern Medical School at the University of Texas Health Science Center at Houston and the Memorial Hermann-Texas Medical Center
  • Anthony L. Estrera, MD: Cardiothoracic and Vascular Surgeon, McGovern Medical School at the University of Texas Health Science Center at Houston and the Memorial Hermann-Texas Medical Center
  • Ara A. Vaporciyan, MD: Thoracic Surgeon, University of Texas MD Anderson Cancer Center

Disclosures
None to report.

Funding/Support
None to report.

Ethical Approval
Reported as not applicable.


References

  1. Ahmadi N, McKenzie ME, MacLean A, et al; Evidence-Based Reviews in Surgery Steering Group. Teaching evidence based medicine to surgery residents—is journal club the best format? A systematic review of the literature. J Surg Educ. 2012;69(1):91-100. https://doi.org/10.1016/j.jsurg.2011.07.004
  2. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225-1230. https://doi.org/10.1016/S0140-6736(03)14546-1
  3. Kitto S, Petrovic A, Gruen RL, Smith JA. Evidence-based medicine training and implementation in surgery: the role of surgical cultures. J Eval Clin Pract. 2011;17(4):819-826. https://doi.org/10.1111/j.1365-2753.2010.01526.x
  4. Deenadayalan Y, Grimmer-Somers K, Prior M, Kumar S. How to run an effective journal club: a systematic review. J Eval Clin Pract. 2008;14(5):898-911. https://doi.org/10.1111/j.1365-2753.2008.01050.x
  5. Kelly AM, Cronin P. Setting up, maintaining and evaluating an evidence based radiology journal club: the University of Michigan experience. Acad Radiol. 2010;17(9):1073-1078. https://doi.org/10.1016/j.acra.2010.04.021
  6. Linzer M. The journal club and medical education: over one hundred years of unrecorded history. Postgrad Med J. 1987;63(740):475-478. https://doi.org/10.1136/pgmj.63.740.475
  7. Linzer M, Brown JT, Frazier LM, DeLong ER, Siegel WC. Impact of a medical journal club on house-staff reading habits, knowledge, and critical appraisal skills: a randomized control trial. JAMA. 1988;260(17):2537-2541. https://doi.org/10.1001/jama.1988.03410170085039
  8. Burstein JL, Hollander JE, Barlas D. Enhancing the value of journal club: use of a structured review instrument. Am J Emerg Med. 1996;14(6):561-563. https://doi.org/10.1016/S0735-6757(96)90099-6
  9. Letterie GS, Morgenstern LS. The journal club: teaching critical evaluation of clinical literature in an evidence-based environment. J Reprod Med. 2000;45(4):299-304.
  10. Nasca TJ, Philibert I, Brigham T, Flynn TC. The Next GME Accreditation System—rationale and benefits. N Engl J Med. 2012;366(11):1051-1056. https://doi.org/10.1056/NEJMsr1200117
  11. Yang SC, Merrill W; for Thoracic Surgery Milestone Working Group. Educational Milestone development in Phase II specialties: thoracic surgery. J Grad Med Educ. 2014;6(1)(suppl 1):329-331. https://doi.org/10.4300/JGME-06-01s1-14
  12. Lee AG, Boldt HC, Golnik KC, et al. Using the journal club to teach and assess competence in practice-based learning and improvement: a literature review and recommendation for implementation. Surv Ophthalmol. 2005;50(6):542-548. https://doi.org/10.1016/j.survophthal.2005.08.002
  13. Lee AG, Boldt HC, Golnik KC, et al. Structured journal club as a tool to teach and assess resident competence in practice-based learning and improvement. Ophthalmology. 2006;113(3):497-500. https://doi.org/10.1016/j.ophtha.2005.12.010
  14. What is a systematic review? Bandolier Web site. http://www.bandolier.org.uk/painres/download/What%20is%202009/What_is_syst_rev.pdf. Published April 2009.
  15. Methodology Checklist 1: systematic reviews and meta-analyses. SIGN Web site. http://www.sign.ac.uk/methodology/checklists/20150417SRchecklist.doc
  16. Critical appraisal: notes and checklists. SIGN Web site. http://www.sign.ac.uk/methodology/checklists.html
  17. Hartzell JD, Veerappan GR, Posley K, Shumway NM, Durning SJ. Resident run journal club: a model based on the adult learning theory. Med Teach. 2009;31(4):e156-e161. https://doi.org/10.1080/01421590802516723
  18. Rand DG, Nowak MA. Human cooperation. Trends Cogn Sci. 2013;17(8):413-425. https://doi.org/10.1016/j.tics.2013.06.003


Citation

Antonoff MB, Nguyen TC, Luc JGY, et al. A structured, debate-style cardiothoracic surgery journal club for trainee acquisition and application of seminal literature. MedEdPORTAL. 2016;12:10521. https://doi.org/10.15766/mep_2374-8265.10521

Received: May 26, 2016

Accepted: December 12, 2016