Research Claims Are Skirting the Peer Review Process

Marcia Frellick

February 24, 2015

Major claims have been reported in the media about Theranos, a privately held biotechnology company that has developed novel approaches for conducting diagnostic tests from a single drop of blood.

These claims have been reported in The Wall Street Journal, Business Insider, Forbes, Fortune, Silicon Valley Business Journal, San Francisco Business Times, and Medscape Medical News. However, they do not appear in peer-reviewed journals and have not been subjected to the rigor of traditional scientific screening, John Ioannidis, MD, DSc, from the Stanford Prevention Research Center in California, writes in a viewpoint article published in the February 17 issue of JAMA.

Theranos, which has teamed up with Walgreens pharmacies to establish Theranos Wellness Centers, is just one example of a company that promotes health solutions without having to disclose how its technology works, Dr Ioannidis says. But stealth research is widespread, he notes, and it poses a conundrum: How can the validity of a company's claims be assessed without peer review? Traditionally, with peer review, innovators would have to convince and satisfy mainstream reviewers who represent the very thinking the innovators aim to disrupt.

"[S]tealth research creates total ambiguity about what evidence can be trusted in a mix of possibly brilliant ideas, aggressive corporate announcements, and mass media hype," writes Dr Ioannidis.

Among the claims he cites for Theranos are that the company can run hundreds of tests from one drop of blood instead of whole vials and that it can do it quickly and more cheaply than in traditional laboratory tests.

Dr Ioannidis notes, however, "There is no mention of overdiagnosis, false-positive findings, or the potential for escalation of iatrogenic disease secondary to misplaced and perhaps overly zealous diagnostic and screening efforts."

When Is Secrecy Warranted?

Stealth research is not new, Dr Ioannidis points out. The nuclear bomb and satellite technology, for instance, were developed without full disclosure to the scientific community for military reasons. But is stealth research necessary in health innovation?

Dr Ioannidis suggests paths to make scientific literature more receptive to innovators.

"This could include models in which reports of disruptive discoveries that are in dissonance with the mainstream can still be communicated as preprints without prior peer review." In that case, it must be made clear to the researchers and the public that no peer review has taken place. That would open up conversation about the innovation, he says.

Debra Mathews, PhD, assistant director of science programs at the Johns Hopkins Berman Institute of Bioethics in Baltimore, Maryland, says many companies are using stealth research. She told Medscape Medical News the incentives need to be changed for private companies to increase their data sharing.

In regular translational science, if you get funding from the National Institutes of Health, you have to publish that work, and because taxpayers pay for the work, the public gets to see the methodologies, the data sharing, and the outcome. If you have private funding, a company is accountable only to its investors, and the incentives have to come in different ways, Dr Mathews says.

She added that there are ways to change incentives and increase data-sharing requirements at the pressure points: the Food and Drug Administration, if a company wants to bring the product to market; the US Patent and Trademark Office, if it wants intellectual property protection; and the Centers for Medicare & Medicaid, if it wants these products or services to be widely covered.

"The fact that there are these parallel systems is fine and good for human health... In private companies [however], the incentive structures are out of step with what ought to be the goal for improving human health and reducing human suffering when you're working in the field of biomedicine," she says.

Academics must meet rigorous demands to make sure the science holds up every step of the way. It is a necessary system of checks and balances, she says.

"Peer review isn't perfect, but like democracy, it's the best thing we have." Without it, you cannot properly evaluate the data, she says.

Dr Ioannidis put it this way: "Biomedical innovation and discovery based on research and development by private and public companies and institutions are essential for advancing medical science and improving clinical care. However, unless stealth research adopts more scientific transparency, investors, physicians, patients, and healthy people will not be able to judge whether some proposed innovation is worth $9 billion, $900 billion, or just $9 — let alone if the innovation will improve the health and well-being of individuals."

Dr Ioannidis and Dr Mathews have disclosed no relevant financial relationships.

JAMA. 2015;313:663-664. Extract


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: