Peer Reviewer Development Practice – a journey to confidence!

Add a comment

By Andrew P. Traynor, PharmD, BCPS

This post is part of an Anniversary Series celebrating the 10-year anniversary of Currents in Pharmacy Teaching and Learning (CPTL) and the two-year anniversary of CPTL Pulses. In this Series, editorial board members are describing and celebrating milestones in our development and advances in support of publishing in pharmacy education.

“One important key to success is self-confidence.

An important key to self-confidence is preparation.”

– Arthur Ashe

From my first experiences with medical literature, publication, and mentored peer review to the present day of continued publication and developing peer review skills in learners, I have felt and observed the full spectrum of emotions including joy, anger, surprise and sadness. Perceptions of peer review seem to change regularly, ranging from “What’s the point?” to “This is extremely valuable!” The literature reflects this struggle, and it often feels like peer review invitations are declined in favor of primary role responsibilities.1 Regardless, authorities in publishing and journal editing continue to uphold peer review as a vital component of the scholarship process.2,3 What can a journal do to help?

Five years ago, CPTL developed a strategic priority to focus on peer reviewer development. The “why” was to advance the quality of articles published and foster improved experiences amongst authors, reviewers and editors. Through shared values of quality and participants’ professional growth, CPTL hoped the peer reviewer pipeline would be stimulated. Interested faculty joined the board, an Associate Editor for Peer Reviewer Development position was developed, and a board subgroup was organized. Using evidence to inform work, our team stepped back to examine the literature around peer reviewer development to enhance manuscript quality. We identified a lack of clarity around the characteristics that are important for peer reviews and peer reviewers. How could we begin programming without sufficient evidence?

We pursued a research project to identify peer reviewer and peer review characteristics that enhance manuscript quality and editorial decisions. Our team utilized a modified Delphi process with experts being editors from journals with high pharmacy faculty publication frequency.4 In study rounds two and three, experts rated characteristics they identified in round one as either required or helpful for enhancing a manuscript’s quality or making an editorial decision. A small number of characteristics (Table 1) were identified as required.

Table 1: Characteristics Identified as Required To Improve Manuscript Quality and Aid Editorial Decisions

Characteristic

Percent of Editors Rating as Required for Quality/Decision

Peer reviewer is knowledgeable in the content area they are reviewing.

94%/71%

Peer reviewer provides specific feedback.

88%/65%

The peer review is a critical analysis of the article.

88%/76%

In the third round of the study, we wanted to know more about the indicators of peer reviewer knowledge. Most experts identified two knowledge requirements for peer review quality;

  • Confidence in the area sufficient to provide a peer review (69%)
  • Current understanding of literature in the area (63%)

What gives a person sufficient confidence? Research concludes that self-confidence comes from practice progressing toward a state of mastery or accomplishment.5 Combined with topics/activities of training identified by the editors (Table 2), we created a peer reviewer development program with a practice peer review capstone to enhance confidence. Participants review a previously submitted article in first submission form and compare their feedback and decisions to actual reviews that our editor identified as helpful in advancing quality and a decision. Participants are able to review the multiple paper versions from submission through acceptance to examine the difference made by peer review and the tactics used by peer reviewers in providing guidance.

Table 2: Peer Reviewer Training Elements/Strategies Rated as Very Valuable or High ROI

Training Element/Strategy

Percent of Editors Rating as Very Valuable or High ROI

Information on reviewer expectations

94%

Information on the review process

88%

Hands-on experiences with review activities

88%

Exposure to good and bad reviews

88%

Feedback

81%

Conducting manuscript reviews

82%

Developing knowledge of peer review process and journal guidelines

77%

Asking for feedback

75%

*Bolding highlights the dimensions of training incorporated through the practice peer review capstone.

Quote from Participant

“I appreciated every single table in that slide set showcasing what areas I should comment on, the most important experiences, and how to keep my skills up to par. I also love the checklist to determine whether you’re commenting on all appropriate items. Tools like that are a great way to develop confidence when you’re attempting something for the first time.”

Is the program making a difference? Anecdotally, yes! In an analysis of our program, users accessed practice reviews 821 times in the past six months. If a journey to confidence in peer review is one you or a colleague or learner would like to join, you’re invited to participate by following the directions below.

Directions for Peer Reviewer Development Programming

  1. Go online to concordia.blackboard.com
  2. Click Alternate Login
  3. Enter user name: cuwpharmacy and password: concordia
  4. Under “My Organizations” click CPTL Peer Reviewer Development
  5. On the left hand toolbar, click “Content”
  6. The description at the top and each folder provides further directions for navigating.

Acknowledgements

The success of the peer reviewer development program would not be possible without the visionary support of Robin Zavod and work of Kristin Janke and Andrew Bzowyckyj in developing program content.

References

  1. Kronick DA. Peer review in 18th-century scientific journalism. JAMA. 1990;263(10):1321-1322.
  2. International Committee of Medical Journal Editors. Responsibilities in the Submission and Peer-Review Process. Accessed November 15, 2019.
  3. Committee On Publication Ethics (COPE) Council. COPE Ethical guidelines for peer reviewers. September 2017. Accessed November 15, 2019.
  4. Janke K, Bzowyckyj A, Traynor A. Editors’ Perspectives on Enhancing Manuscript Quality and Editorial Decisions Through Peer Review and Reviewer Development. Am J Pharm Educ. 2017;81(4):73.
  5. National Research Council. 1994. Learning, Remembering, Believing: Enhancing Human Performance. Washington, DC: The National Academies Press. https://doi.org/10.17226/2303.

Andrew Traynor, PharmD, BCPS is the Associate Editor for Peer Reviewer Development with CPTL. He is a professor and chair of pharmacy practice at Concordia University Wisconsin School of Pharmacy. Andy’s educational scholarship interests include leadership development, practice skills development and academic pharmacy administration. Andy is a husband and father of two boys. He enjoys sports and is an active golfer and curler. Because he loves to eat, Andy enjoys cooking and discussing cooking, especially smoking meats.

Pulses is a scholarly blog supported by a team of pharmacy education scholars. 

Leave a comment