Peer Review of Manuscripts: An Online Training Module

Publication ID Published Volume
9444 June 14, 2013 9

Abstract

Peer Review of Manuscripts is an online training module designed to develop and improve the skills of manuscript reviewers. The module includes discussion of the importance, history and process of peer review, an overview of the uniform requirements for manuscripts submitted to biomedical journals, discussion of authorship ethics and reviewer etiquette related to manuscript publication, and a detailed guide for reviewing all the different sections of a manuscript.

Background and Implementation: Publishing one’s work in the peer-reviewed literature is the time-honored tradition of academic scholarship. Publication establishes scholarship by providing a platform through which scholarly work can be presented and disseminated. Publication is used as evidence of academic productivity and as criteria for advancement and promotion in academics. A critical component of publishing is the peer review process in which peers volunteer to review the scholarship of others and to make recommendations to the journal editors and the authors.

There is published literature on what makes a good review for a general medical journal as well as the effects of blinding and unmasking on the quality of peer review (Bryne 2000, Oxman 1994). However, when health professions faculty agree to provide peer review of manuscripts, either to editorial boards of scientific journals or simply to their own peers prior to journal submission, they may feel they do not have the proper skills for adequate and fair review. Believing that these skills can be learned, we created training materials.

Peer Review of Manuscripts was built on the foundation of the Checklist of Review Criteria for Research Manuscripts, one of the documents in a monograph series prepared by a Joint Task Force of Academic Medicine and the Group of Educational Affairs of the Association of American Medical Colleges and published in Academic Medicine in 2001 (Bordage et al. 2001). Peer Review of Manuscripts includes discussion of the importance, history and process of peer review, an overview of the uniform requirements for manuscripts submitted to biomedical journals, discussion of authorship ethics and reviewer etiquette related to manuscript publication, and a detailed guide for reviewing all the different sections of a manuscript.

Peer Review of Manuscripts was originally created for use with the curriculum of the Academic Pediatric Association’s Educational Scholars Program (APA-ESP), a 3-year faculty development program for junior faculty in academic pediatrics. (Note: the content of the module is not pediatric-specific, but is appropriate for any health professions faculty reviewing manuscripts for any scientific journal.) The module was created for the APA-ESP unit on Peer Reviewing, as a self-instructional introductory overview. In addition to this module, other unit activities include application of skills through a simulated practice review, a “real” review of a manuscript submitted to a journal, and telephoned small group conversations with other Scholars also reviewing manuscripts. Through a relationship with the editors of Academic Medicine, Pediatrics, and Academic Pediatrics, Scholars serve as ‘apprentice reviewers,’ assigned to review an actual manuscript submitted for consideration and provided feedback on their review submission.

The APA-ESP Scholars valued the learning experience and the journal editors applauded the quality of reviews from most of these novice reviewers. Editors of two of these journals requested the availability of these materials for broader use. Therefore, we made a decision to make this module available to a wider audience. We obtained permission from the publisher of Academic Medicine to publish this work built on the foundation of the Checklist of Review Criteria for Research Manuscripts.

Effectiveness of the Work: At the time of this submission, this Peer Review of Manuscripts module has been used with two cohorts of the Academic Pediatric Association’s Educational Scholars Program (n=9 completed in 2010-11; n=16 currently ongoing for 2011-12). Scholars’ evaluation of the Peer Reviewing Unit has been very positive.

Of the first cohort, all nine participants successfully completed the module. The ratings by the six scholars who returned completed surveys are as follows:

Did you participate in this activity?

(n=Yes) 6

Did it stimulate your interest? 4

1=Low

5=Very High

Time required 6.6 hours ( range 1-15)

for activity

(Hours)

Value received 3.9 ( range 2-4)

1=Low

5=Very High

Participants have especially valued the navigability of the module and the bite-size components of the content. Participants found the practice reviews and the “live” reviews very helpful to solidify their newly developing reviewing skills and their confidence in their contribution to academics; and they valued the opportunity for synchronous discussion by phone with others who were also learning the content and practicing new-found manuscript reviewing skills. And finally, many participants recognized the value that peer reviewing has for making them better authors themselves.

From the beginning, journal editors from Academic Pediatrics and Pediatrics enthusiastically supported our efforts to develop better reviewers. These editors participated by assigning the Scholars as ‘apprentice reviewers’ to review actual manuscripts submitted to their journal for consideration. Editors from both journals commented on the quality and depth of the reviews submitted. After the first cohort completed the unit, at least three of the nine Scholars were made regular reviewers by the collaborating journals. By the second cohort in 2011, enthusiasm for and confidence in the Peer Review of Manuscripts module had steadily increased such that a third journal editor, from Academic Medicine, had also committed to providing assignments for actual journal reviews.

Limitations of the Resource, Possibilities for Improvement/Expansion: Although Peer Review of Manuscripts is a self-learning module, there may be a limitation of simply reading the module alone. The effectiveness of the module is increased by linking the module with other activities (e.g. practice reviewing of actual or simulated articles or article sections, discussions of the reviewing process). Having a faculty facilitator as a guide or sounding board, could make the process of learning to review more efficient and effective.

Although designed for review of written manuscripts, the content might be adaptable to help learn the necessary skills for reviewing other forms of scholarship submitted for peer review (e.g. online learning objects such as navigable modules, podcasts, videos). 

Citation

Chandran L, Niebuhr V. Peer review of manuscripts: an online training module. MedEdPORTAL Publications. 2013;9:9444. http://doi.org/10.15766/mep_2374-8265.9444

Educational Objectives

  1. To access and use the uniform requirements for manuscripts submitted to biomedical journals.
  2. To describe the peer review process for manuscripts.
  3. To discuss authorship ethics related to manuscripts.
  4. To use the Checklist of Review Criteria for Research Manuscripts from joint task force of Academic Medicine and GEA-RIME.
  5. To access and use three useful resources for manuscript reviewers.
  6. To demonstrate responsibilities and etiquette of reviewers.

Keywords

  • Peer Review, Publishing

References

This module currently resides on the Academic Pediatric Association website at http://www.academicpeds.org/esppeerrev/page_01.htm.

Material Access

Please sign in to access this material.

Please register for an AAMC account if you do not have one.

Register