Is it Time to Embrace Prepublication Study Results?

Marcia Frellick

December 17, 2015

Researchers and editors from three prominent research institutes say it is time for clinical research to follow the lead of other disciplines and open clinical research to prepublication review.

Michael S. Lauer, MD, deputy director of extramural research at the National Institutes of Health (NIH) in Bethesda, Maryland; Harlan M. Krumholz, MD, from the Department of Medicine at Yale University in New Haven, Connecticut; and Eric J Topol, MD, director of the Scripps Translational Science Institute in La Jolla, California, and editor-in-chief of Medscape Medical News, outlined their argument in a commentary published in the December 17 issue of the Lancet.

The authors list time savings as the most obvious advantage of printing results before peer review. Under the current system, it can take months or years for work to get published, they say.

For example, they note that the NIH issued a press release and held a press conference in September about practice-changing data from the Systolic Pressure Intervention Trial (SPRINT) trial after the trial was stopped early in light of substantial positive results. However, the NIH did not release the main results, and the public had to wait for publication of the final paper, actions Dr Topol and Dr Krumholz protested in a New York Times editorial.

The New England Journal of Medicine, which published the final paper in late November, responded in an editorial: "We were...surprised by the call from Topol and Krumholz for immediately 'placing the data on the NIH website.' We believe that it is critical to give the investigators, on behalf of the study participants, who invested years of their lives in the study, the opportunity to see what led the sponsor to stop the trial and then the opportunity to distill a clinical message from it. There are cogent reasons to follow this approach rather than put trial data in the public domain before those who gathered the data have had a chance to analyze it."

Dr Lauer and colleagues note that other disciplines, including mathematics, astronomy, and computer science, have been posting manuscripts since the 1990s on publicly available websites such as arXiv, where researchers get rounds of voluntary peer review before submitting the work to a journal.

"Other scientific fields have come into the internet era, in which timely, global conversations about rapidly evolving scientific discoveries are stimulated," the authors conclude. "Clinical research has lagged far behind — it is time for us to catch up."

Attempt in 1990s Was "Too Early"

Richard Smith, MD, longtime editor of the BMJ until 2004, told Medscape Medical News he agrees that clinical research is behind the times, "perhaps because the big journals are making huge profits that their owners are anxious not to risk."

He said the BMJ tried teaming up with Stanford University Libraries to offer preprinting in 1999 with an electronic archive called, where authors could post their findings for free before, during, or after peer review by other agencies as soon as studies were complete.

"The experiment didn't work," he said. "It was too early." He said there were very few submissions then, because clinical researchers were too nervous about posting preprints. Reasons, he said, included worries they wouldn't get "proper" publication, that traditional peer review would guard against dangerous ideas that could injure patients, and that their ideas would be stolen.

Dr Smith now consults with F1000Research, a preprint server Dr Lauer and colleagues mention in the commentary. According to information on the website, the platform "publishes articles, posters and slides reporting basic scientific, translational and clinical research within the life sciences and medicine. F1000Research is a scholarly publication platform set up for the scientific research community; each article has at least one author who is a qualified researcher or clinician actively working in the life sciences and who has made a key contribution to the article."

With such platforms, Dr Smith says, "in some ways the future is already here." He says he favors "postpublication peer review, meaning the whole world having the chance to review a paper. I've long argued that this is the real peer review anyway."

Worth the Wait, Journal Editor Says

Dr Lauer and colleagues argue in their commentary that there can be a cost to society when publication of important research findings is delayed. However, Rita Redberg, MD, professor of medicine at the University of California at San Francisco, and editor of JAMA Internal Medicine, says the wait for peer review is usually a few months and is necessary.

"I believe there is great value to peer review, particularly for papers which involve clinical medicine, as the risks of getting things wrong are high," she told Medscape Medical News. "I think the 2 to 3 months it takes for a paper to be peer reviewed is valuable and well worth that time. Peer review allows review of the statistical methods, ensuring proper analysis."

She says as an editor she has seen papers improve after peer review; for example, statistical significance can change, and an appropriate outcome can be highlighted.

"The paper may change from highlighting a positive secondary outcome to acknowledging the negative primary outcome. In observational studies, suggestions of causation can be appropriately eliminated. And of course journals can offer editorial commentary from experts in the field."

Dr Krumholz has disclosed that he is editor of Circulation: Cardiovascular Quality and Outcomes; and is editor of Journal Watch Cardiology. He receives support from the American Heart Association and the Massachusetts Medical Society, nonprofits that own medical journals. He is the recipient of research agreements from Medtronic and from Johnson & Johnson (Janssen), through Yale University, to develop methods of clinical trial data sharing and chairs a cardiac scientific advisory board for UnitedHealth. Dr Lauer and Dr Topol have disclosed no relevant financial relationships; Dr Topol is editor-in-chief of Medscape Medical News. The views expressed are those of the authors and do not reflect the official positions of the NIH or the US Federal Government. Dr Smith was editor of the BMJ until 2004. He is a paid consultant for F1000 and has a pension from the British Medical Association, an organization which depends to some extent on BMJ Group profits. Dr Redberg is editor of JAMA Internal Medicine but declares no other relevant financial relationships.

Lancet. 2015;386:2447-2449. Extract


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.