Reporting Bias Rife in Antidepressant Studies for Anxiety

Pam Harrison

April 07, 2015

The vast majority of studies evaluating antidepressant efficacy for the treatment of anxiety disorders that are judged by the US Food and Drug Administration (FDA) as positive are published. However, it's a different story for nonsignificant studies, results of a new meta-analysis show.

Investigators at the University Medical Center Groningen, the Netherlands, found that trials that the FDA deemed positive were five times more likely to be published compared with trials deemed not positive. Furthermore, they found evidence for study publication bias (P < .001), outcome reporting bias (P = .02), and spin (P = .02).

The findings, investigators say, provide a skewed view of antidepressant efficacy.

"Antidepressants have value in treating anxiety disorders, but they are not 'miracle drugs,' and there is less evidence to support their efficacy than what appears from the published studies," Annelieke Roest, PhD, told Medscape Medical News in written correspondence.

"It is important that physicians have a realistic view of the efficacy of these drugs so they can make a well-justified decision when prescribing these medications," she said.

The findings were published online March 25 in JAMA Psychiatry.

Skewed View?

The investigators note that there is "strong evidence that significant results from randomized clinical trials are more likely to be published than non-significant studies.

"As a consequence, published studies, including meta-analyses, may overestimate the benefits of treatments while underestimating their harms, thus misinforming physicians, policy makers, and patients."

The researchers examined reporting biases in double-blind, placebo-controlled trials on the pharmacologic treatment of anxiety disorders and quantified the extent to which these biases inflate estimates of drug efficacy.

They identified phase 2 and 3 double-blind, placebo-controlled trials registered with the FDA and conducted in pursuit of marketing approval of second-generation antidepressants for the treatment of generalized anxiety disorder, panic disorder, social anxiety disorder, posttraumatic stress disorder, and obsessive-compulsive disorder.

Studies involving seven selective serotonin reuptake inhibitors and two serotonin norepinephrine reuptake inhibitors were evaluated.

Of the 57 trials identified by researchers, findings from 41 of them (72%) were judged to be positive by the FDA.

However, 43 of 45 (96%) of the articles published in the literature were positive (P < .001).

This suggests that trials the FDA determined to be positive were five times more likely to be published in agreement with that determination compared with trials determined to be not as positive by the FDA (risk ratio, 5.20; 95% confidence interval, 1.87 - 14.45; P < .001).

Sixteen of the 57 trials identified by investigators (28%) were not judged to be positive by the FDA, and seven of these trials (44%) were not published.

In contrast, only one of the 41 positive trials (2%) was not published. The difference between the not-positive and positive trials was significant (P < .001)

Of the 16 not-positive trials, only 3 (19%) were published in agreement with the FDA, investigators add ― in other words, they were published without a positive conclusion in the literature.

Importantly, in 3 of the 16 not-positive trials, effects were reported as statistically significant in the journal article, leading to a positive conclusion that conflicted with the FDA review of the study.

"By contrast, outcome reporting bias was found in none of the 41 FDA-positive trials," investigators add. The difference between the not-positive and positive trials was statistically significant (P = .02).

Spin was present in an additional 3 of 16 (19%) of the not-positive trials as well; it was not present for any of the positive trials.

Each of these three trials reported that the primary endpoint was nonsignificant in the results section of the articles in which they were published, but in the abstract, it was concluded that the trial was positive, the investigators note.

The FDA classified one of these three trials as "questionable" and the other two as "negative."

When investigators compared the overall treatment effect from a meta-analysis of the literature to the treatment effect from a meta-analysis of the FDA reviews, there was an overestimation of the treatment effect of 15%, as determined on the basis of the literature analysis, a difference that was not statistically significant.

This is in contrast to a previous study (N Engl J Med. 2008;358:252-260) that showed a larger overestimation ― one of 32% ― for the efficacy of antidepressants in the treatment of depressive disorder, as determined on the basis of literature reports.

"These medications have been shown to have an overall meaningful effect on anxiety, as has been shown for nonpharmacological treatments, like cognitive behavior therapy, as well," Dr Roest said.

"But they will not have a positive effect on all individuals, and physicians should carefully consider and discuss the various treatment options with their patients in order to determine the best treatment effect for each of them."

Antidepressants Not Very Useful

Asked by Medscape Medical News to comment on the study, Irving Kirsch, PhD, associate director, Program in Placebo Studies and the Therapeutic Encounter, Harvard Medical School, Boston, Massachusetts, disagreed with Dr Roest's statement that antidepressants have "an overall meaningful effect" on anxiety.

Dr Irving Kirsch

"In fact," he said, "the overall effect size reported in the study [0.38; 95% CI, 0.33 - 0.42] is very small and may not even be detectable by treating physicians."

At the same time, Dr Kirsch felt that the Dutch group's data were reliable, because they mirrored the effect size (0.27) that he and his colleagues found in a pre- and postmarketing study of paroxetine (multiple brands) in the treatment of anxiety.

"This is an effect size that one sees for these drugs in the treatment of depression," Dr Kirsch observed.

"And they are much smaller than what the National Institute of Clinical Health and Excellence has suggested would be a clinically meaningful effect size, which they have posited should be 0.5," he said.

"So antidepressants are as useful in anxiety disorders as they are in depression, which is not very."

The study was supported by a grant from the Dutch Brain Foundation. Dr Roest and Dr Kirsch report no relevant financial relationships.

JAMA Psychiatry. Published online March 25, 2015. Abstract


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.