More than half of clinical trial abstracts published in top psychiatry and psychology journals that were negative for the primary outcome exaggerated the clinical significance of a particular treatment, according to a new analysis.
This is potentially "dangerous," first author Samuel Jellison, third-year medical student, College of Osteopathic Medicine, Oklahoma State University Center for Health Sciences in Tulsa, told Medscape Medical News.
"If doctors are being misled by researchers, it could lead to changes in their clinical practice that are scientifically unfounded and, maybe, a decline in patient care," said Jellison.
He hopes the study sheds light on this "poor research practice" and strengthens the ability of clinicians to "critically assess medical literature."
The study was published online August 5 in BMJ Evidence-Based Medicine.
Prevalent, Concerning
The investigators combed PubMed for randomized controlled trials of psychiatric and behavioral treatments published between 2012 and 2017 in six top psychology and psychiatry journals: British Journal of Psychiatry; Psychological Medicine; Journal of the American Academy of Child and Adolescent Psychiatry; JAMA Psychiatry; Journal of Child Psychology and Psychiatry; and American Journal of Psychiatry.
Of the 486 trials examined, 116 had nonsignificant primary endpoints and were included in the analysis.
"Spin" was defined by the investigators as the "use of specific reporting strategies, from whatever motive, to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results."
The researchers found evidence of spin in the abstracts of 65 (56%) of the published trials. Spin was found in 2 (2%) titles, 24 (21%) abstract results sections, and 57 (49%) abstract conclusion sections. In 17 (15%) trials, spin was identified in both the results and conclusion sections of the abstract.
"So not only does spin tend to be prevalent within the literature, but it also may be prevalent within a single article," said Jellison.
Spin was more common in trials that compared a particular drug/behavioral therapy with a placebo intervention or usual care. Industry funding was not associated with a greater likelihood of "spin" results.
Jellison said the most common types of spin were found in the conclusion section of the abstract, which usually manifested itself in one of three ways:
Authors had multiple prespecified primary outcomes of their study but only found statistically significant results for one of the primary outcomes. They proceeded to focus on the significant outcome, neglected to mention the statistically nonsignificant outcome(s), and claimed that their trial was successful and their treatment beneficial.
Authors had a statistically nonsignificant primary outcome but a statistically significant secondary outcome, thus claiming their treatment beneficial based on the significant secondary outcome while failing to disclose the nonsignificant primary outcome.
Authors prespecified a superiority trial but after finding statistically nonsignificant results for their primary outcome, they claimed "equivalence" or "non-inferiority" for their treatment vs the comparator treatment.
"Spin is a prevalent problem that has infiltrated the literature of many specialties," said Jellison. "Other studies have shown spin to be equally prevalent within other specialties, making this a problem for clinicians in all specialties, not just psychiatry."
The researchers acknowledge the findings may not be widely applicable to clinical trials published in all psychiatry and psychology journals and — despite the use of objective criteria to define spin — their assessments, inevitably, are subjective.
"Authors, journal editors and peer-reviewers should continue to be vigilant for spin to reduce the risk of biased reporting of trial results," they conclude.
"Sadly Consistent"
Commenting on the results for Medscape Medical News, Lisa Cosgrove, PhD, clinical psychologist and professor at the University of Massachusetts Boston, said, "This is an important and concerning study as busy clinicians all too often only read the abstract." It is also "sadly consistent with previous studies."
Cosgrove noted that while this research team did not find an association between industry funding and spin, a recent Cochrane study did.
She also noted that the study by Jellison and colleagues shows that guild interests as well as financial conflicts of interest (FCOI) can exert "undue influence on the medical/scientific literature."
"Guild interests refer to the fact that any medical/scientific specialty group is vulnerable to implicit bias," Cosgrove said. "It is part of the human condition to have implicit biases — and be blissfully unaware of them."
Editor's note: This article has been revised to clarify that the half of studies that exaggerate the findings are only those with negative results, not all psychiatric journal study abstracts.
The study had no specific funding. Jellison and Cosgrove have disclosed no relevant financial relationships.
BMJ Evid Based Med. Published online August 5, 2019. Full text
For more Medscape Psychiatry news, join us on Facebook and Twitter
Medscape Medical News © 2019
Cite this: 'Spin' Prevalent in Top Psychiatry Journal Abstracts - Medscape - Aug 06, 2019.
Comments