COMMENTARY

Society Introduces Template for Peer Reviewers Uneasy With Stats

Andrew J. Vickers, PhD

Disclosures

April 26, 2011

New York – Following widespread concerns about the quality of peer review, the International Society of Medical Journal Peer Review (ISMJPR) today released a "template" that journal editors can provide to reviewers. In a statement, Dr. Eric Hamadi, Chair of ISMJPR, stated that the template was a "major step forward for ensuring consistency in reviewing" and that it would provide a "bonus to the scientific process." The template, which codifies contemporary reviewer behavior, is given below.

The Template

The authors should be congratulated for their attempt to study ___. Unfortunately, the paper is beset by several major flaws.

  1. A number of points can be raised that are entirely tangential to the main thrust of the paper and have no bearing on the conclusions. However, discussing these points in my review demonstrates my deep and nuanced understanding of the subject matter. For example, although __________ would not affect the findings, mentioning it illustrates my methodologic sophistication and thorough clinical knowledge.

  2. In my considered opinion, __________ constitutes a major flaw/limitation/point requiring elaboration. It is true that the authors have already addressed this point in some detail, but I am extremely busy -- indeed, I head to the airport for an important meeting in a few hours -- and certainly do not have time to read papers for peer review thoroughly.

  3. I have extremely limited knowledge of statistics, and this continues to be a major embarrassment to me. But I am acutely aware that facility with statistics is pretty much essential for any medical researcher wishing to be taken seriously. Therefore it is incumbent upon me to make a comment on the statistical approach taken by the authors. Accordingly, I would like to mention __________ as a statistical weakness. I know that this is irrelevant/wrong/nonsensical, but I would hate it to appear as though I was entirely baffled by the statistics section of the paper.

  4. One claim made by the authors is __________ and I have previously held a contrary position. I must therefore make some arguments against this point as follows: __________. These arguments are illogical and unfounded, but it is essential that anything casting doubt on a published paper of mine is kept out of the literature. The authors need to keep my reputation in mind when writing up the results of their research.

  5. There is an error in __________. Actually there isn't, but I scanned through the paper quickly and it looked like a mistake. Anyone looking at this part of the paper would work out that it was correct in about 5 seconds, but, well, I didn't spend 5 seconds.

  6. Here is a counterargument to the main findings: __________. Now I have not even started to think through the implications, such as the fact that it affects so few patients and/or has such a marginal effect that it will make only a trivial difference to the main results. A flaw is a flaw, and I need to point it out, irrespective of its scientific import.

  7. I am aware that the authors have spent months thinking about the paper, whereas I have only thought about it for the past 5 minutes. Nonetheless, it is essential that the authors modify table ___, delete figure ____, modify the analysis of ________, and change the sensitivity analysis to adjust for _______ rather than ___. I realize that there are pros and cons to all of these points, such that taking my advice might well weaken the paper, but that is hardly my concern.

  8. The authors should describe ____, _____, and _____ in more detail; add some paragraphs discussing ______; give a figure plotting ___ against ____; add a new section mentioning ____ as a limitation; and provide new tables splitting the data set by age, gender, and baseline function.

  9. The paper is too long and needs to be considerably shortened; in particular, there are too many tables and figures.

  10. At this point, I would like to make a statement that has absolutely nothing to do with the paper. However, pointing out _____ does allow me to make some condescending remarks about the authors' general lack of scientific and clinical knowledge. I, after all, am the expert as evidenced by the fact that the editor considers me important enough to be the gatekeeper to the sacred text of their journal.

  11. The authors not only fail to demonstrate a cure for metastatic pancreatic cancer but do not predict the winner of next year's World Series. These are major omissions that should be addressed in any resubmission.

With apologies to The Onion.

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as:

processing....