MIST Published: Editorial Highlights Discrepancies, Implications for Other Migraine Trials

Shelley Wood

March 11, 2008

March 11, 2008 (Dallas, TX) - When they were first presented in 2006, results from the Migraine Intervention with Starflex Technology (MIST) trial suggested that patent foramen ovale (PFO) closure did not cure headache but that headache days and headache burden were significantly reduced in device-treated patients as compared with sham-treated controls. Published online March 3, 2008 in Circulation [1], the MIST results show no difference for any of the trial's main end points, and, conspicuously, the co-principal investigator for the trial who first raised concerns about how the study sponsor was managing the data is not listed as an author.

An accompanying editorial points out numerous problems with the design and execution of the trial and says the ongoing MIST controversy will likely have an impact on other migraine/PFO trials [2].

MIST co-PI Dr Peter Wilmshurst (Royal Shrewsbury Hospital, UK) told heartwire last year that he had commented extensively on an early draft of the paper but for a number of reasons, including the fact that the paper made no mention of a review of the echocardiography data that he himself had conducted, refused to sign off on the version ultimately submitted. Wilmshurst also alleges that he was never given access to the patient-level data from the study and had concerns about an exploratory post hoc analysis that excluded two "outlying" patients from an analysis of migraine-days per month.

A Circulation spokesperson confirmed that neither Wilmshurst nor his neurologist colleague at Shrewsbury, Dr Simon Nightingale, was listed as author when the paper was first received by the journal. The two men are also absent from the names listed in the paper describing the trial's steering committee, although both were originally steering-committee members.

Speaking to heartwire , Dr Michael J Mullen (Royal Brompton, London, UK), a cardiologist and one of the paper's authors, defended the publication.

"We were keen that they would be authors on the paper, and there was no intention of ever excluding them, but it was clear that they were unwilling to sign off to authorship," Mullen explained. "To a certain extent, the whole process was being held ransom, and we felt there was a duty, most importantly to the patients who took part in the study and to the scientific community, to have the data from this study in print. And that is why we proceeded."

The effervescence of positivity

In an editorial accompanying the MIST paper in Circulation, Dr John Carroll (University of Colorado, Aurora) argues that the now-nonsignificant results of MIST are at the core of what he calls the "ongoing MIST controversy" and will end up having an impact on how other PFO/migraine trials turn out.

"Interestingly and somewhat disturbingly, MIST is now a completely negative study," he writes. "The de facto retraction of a positive clinical trial previously presented at a high-profile international meeting remains a major issue of concern from both the scientific and ethical perspective. Even in late 2007, experts in the field continued to quote the positive secondary end points of MIST, and it is only now in wake of the published final results that we all appreciate the effervescence of such positivity."

President and CEO of NMT Medical, the trial sponsor, John Ahern, also commenting on the publication of MIST and the accompanying editorial, emphasized that when the results were first unveiled in 2006, the trial results were "clearly presented . . . as preliminary" and that it had failed to reach its primary end point.

"There has been a long delay in the publication of the final results, primarily due to a single investigator, and by no means was the delay an attempt to retract any previous presentation of the data. The remaining investigators were unified in ensuring the data presented in Circulation was fair and accurate, and the results were adjudicated on multiple levels," Ahern stated. "MIST was the first randomized controlled trial to study the relationship between PFO and migraine headache, a groundbreaking trial. Perhaps the focus of the scientific community should not be on any perceived controversies surrounding the trial but on what can be learned from these results to support future research in this area."

Debate building since TCT 2007

As previously reported by heartwire , Wilmshurst and Dr Andrew Dowson (Kings College Hospital, London, UK), a headache specialist, together presented the MIST I results at the ACC 2006 Scientific Sessions, but at the TCT 2007 meeting, Dowson, not Wilmshurst, presented late results from MIST (dubbed MIST III). Wilmshurst, also at TCT 2007, gave a separate presentation outlining his concerns about the MIST trial's conduct and conclusion--he had been unaware that Dowson was presenting any MIST results at the TCT.

After his presentation, Wilmshurst told heartwire that NMT Medical had attempted to marginalize his role in the trial, claiming that he was never a co-PI for MIST; repeatedly blocked his attempts to view the complete MIST data set; lied about whether echocardiograms from the MIST patients had been independently reviewed; and massaged the data to portray its PFO device in the best possible light.

For its part, NMT has insisted that Wilmshurst is on a "warpath" and explained that he had been dropped from the trial for committing unspecified protocol violations. The company has initiated legal action against Wilmshurst and late last year requested, without success, that heartwire remove its earlier coverage of the MIST controversy from www.theheart.org.

Results held ransom

Now, in an interview with heartwire , Mullen acknowledged that he and the other authors believed it was necessary to go ahead with the publication of MIST, despite Wilmshurst and Nightingale's refusal to sign off on the draft being submitted to the journal.

"I signed onto this because I'm absolutely convinced that the data in this paper reflect what went on in the study and had been properly collected; I think the discussion is reasonably well balanced," Mullen said. "We have acknowledged that maybe the methodology wasn't perfect, but if anything, that is the lesson of the study. For anyone to suggest that there is anything more sinister going on is outrageous; it's an insult to the cardiologists, and it's frankly an insult to the patients as well."

Critically, NMT, as well as Dowson, had repeatedly insisted to heartwire that no post hoc review of the residual shunt data in MIST had ever been conducted; Mullen, however, did not deny the existence of the review but was adamant that these data had no place in the paper.

"That review was totally unsafe, and it would have been completely wrong to have included that in any manuscript, in any form, at any time," Mullen told heartwire . "That's not just my opinion, that was the opinion of a number of individuals involved in the study who felt that this was a review that was performed after the fact, by an individual whose career has been based on proving the concept that we set out to prove in the study."

Mullen elaborated that the review had been done without any agreed-upon protocol, after the results were unblinded and known to be negative. "Can you imagine if we found a positive result in that and tried to dress it up as something? How that would have been perceived?" Mullen asked.

Residual shunt review

The MIST trial protocol specified that the implanting interventional cardiologists, and not an independent core lab, would review patients' echocardiograms and determine whether or not there were moderate to large residual shunts, a yes/no question on the patient's data sheet. According to Wilmshurst, who has the emails to prove it, NMT officials instructed MIST investigators to send all patient echocardiograms to Wilmshurst for independent review immediately following the ACC meeting in 2006. Wilmshurst ultimately reviewed the contrast echoes for all but one of the patients who underwent the scan at follow-up. Dowson et al's Circulation paper states that four of 65 patients in whom PFO closure was successful had evidence of a moderate to large residual shunt (6%). Wilmshurst's review of the echocardiograms, however, suggested that the residual shunt rate was far higher--approximately 33%. According to Wilmshurst, the echo data were reviewed a second time by the data monitor, Dr Luke Missault (St Jan Hospital, Bruges, Belgium), who came up with different results but ones that were "statistically consistent" with his own review. While a larger rate of residual shunts might explain why the trial failed, it would reflect negatively on a device that is widely approved outside the US for the treatment of cryptogenic stroke.

In his editorial, Carroll notes that "the lack of an established independent core laboratory for echocardiographic data analysis must haunt the trial investigators."

If anything, dressed-up results are one possibility raised by critics in the years that it has taken MIST to come into print. For one, different aspects of the MIST I results have been reported repeatedly at a range of neurology and cardiology meetings both before and after the 2006 presentation.

Carroll, in his editorial, highlights the fact that trials still in development in 2006 to some degree took their cue from the supposedly positive secondary analyses; this may now prove the wrong tack to have taken.

"It raises concerns about how US trials are going to turn out, because they were designed with a primary end point that was close, if not identical, to the secondary end point in MIST I, which was initially positive," he said. Not only were trial designs shaped by MIST, so too were the type of patients recruited, Carroll observed. "That's certainly why the newer trials will be extremely important to give us the most definitive answer on whether PFO closure is a viable treatment for patients who have more or less medically refractory migraines."

Asked about other controversial aspects of the MIST trial, Carroll said he had endeavored to focus his editorial strictly on the results appearing in the published manuscript, but he conceded that it seemed "unusual" to publish trial results without including the name of one of the trial's principal investigators. As for the questions that have been raised about the residual shunt rates and post hoc analyses, Carroll expressed the belief that these will still continue to be probed in other forums and venues.

His editorial concludes: "Although the sky is certainly not falling in terms of the association between migraines and PFO or the potential benefits of PFO closure in patients with migraines, the controversies surrounding MIST must be appropriately addressed and resolved before any potential benefits of PFO closure in migraine patients can be implied with any level of certainty. In this light, the rush to bring off-label device closure to the migraine sufferer who cannot or will not participate in a clinical trial will hopefully be halted by the publication of MIST."

Published MIST results

In the Circulation paper, Dowson et al report no difference in the primary end point of number of patients with no migraine attacks between 91 and 180 days postprocedure. Results were the same in the per-protocol analysis and in the intention-to-treat analysis (PFOs could not be found or crossed in five of 74 patients). They also saw no differences in the secondary end points, including severity of migraine, change in frequency of migraines, or total headache days. In an "exploratory analysis" that excluded two outliers (two patients in the intervention arm seemed to account for more than one-third of all headache days) the number of headache days was significantly--if modestly--reduced in the implant group (2.2 days per month vs 1.3 days per month; p=0.027.)

In the device arm, there was one case each of cardiac tamponade, pericardial effusion, and retroperitoneal bleed and two cases of atrial fibrillation. In the sham-treated patients, authors reported adverse events mostly related to study medications, including antiplatelet drugs.

In his editorial, Carroll highlighted the high frequency of patients not found to have a PFO during their procedure, calling into question the quality of the echocardiographic screening process; the higher-than-expected rate of serious adverse events in the device-treated patients, raising concerns about the quality of the procedures; and the "unclear number" of residual shunts, raising a red flag about the efficacy of the device itself.

Mullen, for his part, insisted to heartwire that the authors of the Circulation paper took considerable pains to address all the possible explanations for the trial's failure and design shortcomings in the discussion. He also stressed that the authors did not overinterpret the results of the post hoc analysis that excluded two outliers.

"It is reasonable to do these exploratory analyses to look for clues, but obviously you have to be honest, to put it in the context that this was a post hoc analysis, this was not a predefined primary or secondary end point," he said. "And that's in the paper."

Contacted by heartwire about the publication of the MIST results, Wilmshurst declined to comment, citing ongoing legal wrangling with NMT, as well as the fact that he himself has been in touch with Circulation raising his own concerns about the paper.

Mullen, along with Wilmshurst's co-principal investigator Dowson and Dr Richard Peatfield (Charing Cross Hospital, London, UK) are listed in the paper as the "responsible authors." The Circulation paper notes that Mullen (and one other author) have an ownership interest in NMT and that Mullen has received teaching honoraria and acted as a consultant to NMT. Dowson has previously acknowledged receiving funds and personal compensation from NMT, although there is no mention of this in the disclosure paragraph in the Circulation paper. Carroll is an investigator in the ESCAPE trial, St Jude Medical's PFO/migraine study, and he disclosed being a consultant for St Jude, as well as for AGA Medical, another company developing PFO occlusion devices and investigating their role in the treatment of migraine.

  1. Dowson A, Mullen MJ, Peatfield R, et al. Migraine Intervention With STARFlex Technology (MIST) trial. A prospective, multicenter, double-blind, sham-controlled trial to evaluate the effectiveness of patent foramen ovale closure with STARFlex septal repair implant to resolve refractory migraine headache. Circulation 2008; 117:1397-1404. Abstract

  2. Carroll JD. Migraine intervention with STARFlex technology trial. A controversial trial of migraine and patent foramen ovale closure. Circulation 2008; 117:1358-1360.

The complete contents of Heartwire
, a professional news service of WebMD, can be found at www.theheart.org
, a Web site for cardiovascular healthcare professionals.

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as:

processing....