COMMENTARY

OASIS Retraction Exposes Serious Flaws in Clinical Science

John Mandrola, MD

Disclosures

September 14, 2016

If you care about medical science done right, this is a sad turn of events.

When cardiology's leading scientific journal, the Journal of the American College of Cardiology, abruptly retracted the paper reporting results of the OASIS trial, a multicenter comparison of a controversial new technique to ablate atrial fibrillation (AF), patients and doctors lost.

Trust in "the system" was broken.

Let's start with facts:

On the JACC website came this announcement: "This paper has been retracted at the request of the JACC editor in chief, its editorial board, and the JACC ethics board for the following reasons:

  • "In the title and multiple times in the article, the study is referred to as a 'randomized trial,' but deviation from a random allocation of subjects to treatments across sites and the imbalance introduced by a nonrandom 'randomization error' were not disclosed in the manuscript.

  • "Registration with ClinicalTrials.gov was not completed before patient enrollment began."

In 2012, in an editorial on scientific misconduct and errata, Dr Anthony Demaria, then the editor in chief of JACC, referred to retraction as an "action of the greatest gravity" that "should not be undertaken lightly."[1]

Focal impulse and rotor modulation, or FIRM ablation is, or was, a promising technique for identifying key areas of the atria that maintain AF. Preliminary studies suggest ablation at these focal sites improve outcomes for AF ablation.[2] If true and confirmed, this finding would be massive—not only would it improve outcomes of AF ablation, but more important, it would teach us how AF occurs. After a century of investigation, the mechanisms of AF remain unknown.

A company called Topera makes the equipment and software for finding rotors. In 2014, Abbott acquired Topera for many millions of dollars. Abbott now markets the mapping system to doctors like me.

Dr Andrea Natale presented the OASIS trial as a late-breaking clinical trial at the Heart Rhythm Society 2016 Scientific Sessions. Its findings delivered a crushing rebuke for FIRM ablation. OASIS compared FIRM ablation alone, FIRM plus pulmonary-vein isolation (PVI), and PVI alone in patients with advanced AF. Researchers reported that the FIRM-alone arm was stopped early for futility, and the FIRM/PVI group performed worse than PVI alone.

Dr Natale has publicly stated his frustration with the retraction, indicting industry conflicts of interests. On Twitter, he wrote, "Another victory of the industry and their conflicted accolades." In a statement to heartwire from Medscape journalist Patrice Wendling, Dr Natale said that JACC took extreme action when an addendum could have explained the patients excluded for error; they cited other studies published with postrandomization exclusion. As for the infraction of beginning the trial before registration, he referenced a 2012 JAMA study[3] that found only 48% of eligible trials were registered on ClinicalTrials.gov before enrollment.

When asked why they believed industry played a role in the retraction, the authors noted that information in letters to JACC were known only to the investigators and industry; thus, "it [was] obvious that these individuals were acting on behalf and in the best interest of the company."

Comments

This is a big deal. Patients and doctors want to know whether FIRM ablation works or not. More important, though, we need to trust the system of medical evidence.

I'll begin with prepublication review. If there were major flaws in the paper, why weren't they discovered during the editorial/peer-review process?

I'm suspicious that the nonanonymous nature of peer review diminishes scrutiny. Why not send papers out for review without listing the authors? Reviewers need this protection. Although reviews are supposed to be anonymous; academics is a small world. Publication remains the currency of academic progress, so self-preservation rules.

Another problem with the prepublication process is its speed. The temptation of professional societies to tantalize with "late-breaking" clinical trials at medical meetings worsens the problem of cursory reviews. It's not just me who feels this way. Writing in the New England Journal of Medicine, Drs Stuart Pocock and Gregg Stone called the speed with which the landmark hypertension SPRINT trial made it into publication exceptional and surprising.[4]

Then there is the issue of trust in research from Dr Natale's group. PubMed lists more than 400 publications from Dr Natale. He and his coworkers are influential opinion leaders in the field. Will this retraction lead to closer scrutiny of other papers from this group? The clear language translation of JACC's explanation is this: you are experienced researchers; you know what a randomized trial entails; you misled the Heart Rhythm Society and cardiology's leading journal. I've read and reread the authors' response. It is a cogent one.

Now let's consider the authors' contention that industry influence played a role in the retraction decision. Dr Joshua Cooper is the director of electrophysiology at Temple University. He is a respected physician, educator, and researcher. On Twitter, he asked, "There are flaws in every clinical trial. Big ones. If scrutinized, all would be retracted. Why this one? Motivated by industry?"

You can't extract financial conflicts of interest from this story. The findings of OASIS crushed a promising technology. That promising technology stood to make millions of dollars for Abbott. And Abbott supports medical journals and professional societies. These aren't accusations; they are facts.

In response to questions from heartwire , Dr Paul Douglass, chair of the ACC publications and editorial content committee, strongly denied any commercial influence on the publication process. I reached out to Dr Sanjiv Narayan (Stanford University), a cofounder of the Topera rotor mapping system and an associate editor of JACC. He said: "At JACC, we are recused from any discussion in which we may have a real or perceived conflict of interest. Accordingly, I was not involved in any matters at JACC relating to the OASIS study, and I had no input in its review or subsequent discussions at any time."

Yet I'm drawn to Dr Demaria's words in 2012. He cited the Wikipedia entry on retraction: "It indicates that the data and conclusions of the original article should not be used as part of the foundation for future research, implying that the data is erroneous."

I'm not sure this is the case with OASIS and FIRM ablation. The trial had flaws, but it is not the only study (from independent authors) unable to replicate high success rates with FIRM.[5,6]

In summary, I see three possibilities: a fatally flawed trial makes it through the editorial peer-review process of a high-impact journal and is reported as a late-breaking trial at a major medical meeting; an abrupt retraction implicates one of electrophysiology's leading research groups in scientific misconduct; or industry influenced an editorial decision in a scientific journal.

All three are bad. It's impossible to know where the truth lies, and that is the problem.

I'm sad for my field. Once lost, trust is hard to recapture.

JMM

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as:

processing....

Recommendations