Negative Studies Presented Ambiguously at Cancer Conferences

Veronica Hackethal, MD

May 11, 2020

As we approach the American Society of Clinical Oncology 2020 Annual Meeting — one of the biggest cancer meetings in the world, to be held virtually May 29-31 — it is somewhat sobering to read about "spin" disguising negative results.

Researchers sometimes spin negative results in an ambiguous, or even positive, light in their conclusions sections, say a group of Italian authors. Disseminating conclusions that are unjustified by results could be misleading and, in some cases, even harmful in clinical practice, they warn.

The team undertook a systematic review of oral presentations at two major cancer meetings (held by ASCO and the European Society of Medical Oncology [ESMO]) over 3 years. They found that 13% to 47% of studies with negative results were not reported as such. "Despite the formally negative trial result, the authors' conclusions are not negative," they report.   

The study was published online on April 16 in JAMA Oncology.

"Many slides at important meetings like ASCO and ESMO are tweeted or shared on social media in real-time, and this could contribute to the diffusion of questionable scientific messages," said lead author Massimo Di Maio, MD, from the division of medical oncology at Ordine Mauriziano Hospital in Turin, Italy.

"In our analysis we found that, in several cases, authors defined the experimental treatment as an option for clinical practice," Di Maio said.

Some studies failed to show superiority for the experimental treatment but had similar outcomes for the experimental and standard treatments. However, that does not prove non-inferiority for the experimental treatment compared to standard therapy, and it does not mean the experimental treatment may be an option for clinical practice, Maio commented. Proving noninferiority would require a prespecified, clinically "nonrelevant" margin of difference. Using a post-hoc noninferiority interpretation is methodologically questionable, he added.

"If the drug is already available in clinical practice, as is the case for treatments tested in many academic trials, applying that questionable conclusion to clinical decisions could result in patients potentially receiving an inferior treatment," he said.

He added that he is not against presenting negative results, which may have value and deserve attention.  

"I strongly believe that a formally negative or inconclusive result does not imply that the study deserves less attention compared to a positive result," he said. "Negative trials can represent important evidence. They can be used as part of meta-analyses, and can offer important suggestions for further research."

Di Maio said he came up with the idea for this study last year while attending oral sessions at the ASCO 2019 annual conference in Chicago, Illinois. He noticed that several trials with a primary analysis that had negative results were presented with ambiguous or non-negative conclusions. So he discussed the issue with his colleagues back in Italy, and they decided to see how big the problem really is. 

How Big Is the Problem?

The team reviewed oral presentations of phase 3 randomized controlled trials (excluding noninferiority trials) presented at ASCO and ESMO from 2017 through 2019. They separated trials based on whether the primary endpoint had positive or negative results. Then they evaluated the wording of the conclusions sections. Conclusions were considered non-negative when the authors "more or less explicitly" brought up the idea of using the experimental treatment in a clinical setting and did not provide clear conclusions about the negative results.

The analysis included 208 trials, of which 91 had negative primary endpoint results. Among trials with negative results, 29% had non-negative conclusions, of which 22% were presented in 2017, 13% in 2018, and 47% in 2019.

Among the studies with negative results and non-negative conclusions:

  • 50% emphasized a numerically better outcome in the experimental arm despite having a nonsignificant P value (13 cases).

  • 46% emphasized positive results in one or more subgroups (12 cases).

  • 38% emphasized positive results in one or more secondary endpoints (10 cases).

  • 27% used a post-hoc design to interpret noninferiority (seven cases).

The team also compared for-profit and academic (nonprofit) studies and found similar percentages of non-negative conclusions among studies with negative results: 30% (17/57) among nonprofit studies and 26% (9/34) among for-profit studies.

"One could hypothesize that the tendency to present results in a non-negative way is related to economic interests of the study sponsor. However, our results tell us that this is not the case," Di Maio said. "Many researchers tend to present their study as positive, maybe because they are 'in love' with the hypothesis and truly convinced of the efficacy of the treatment they have tested."

Overall, the results imply the need for caution when listening to oral presentations, whether or not the study is sponsored by "big pharma" or a nonprofit.

"Practicing physicians should not read authors' conclusions as pure gold, but should question whether the conclusions are actually supported by the results. I suggest always looking carefully at the study design, primary objectives, and hypothesis," he said.

"We believe that more attention should be paid to the statements included in the conclusions of oral presentations at meetings, and the discussants' role is crucial," the team concludes. In addition, "when the primary endpoint is not met, the word negative should be explicitly used."

Di Maio has reported receiving personal fees from AstraZeneca, Eli Lilly, Bristol Myers Squibb, MSD, Eisai, Takeda, Pfizer, and Janssen. Several coauthors have also reported receiving fees from various pharmaceutical companies.

JAMA Oncol. Published online April 16, 2020. Letter

Follow Medscape Oncology on Twitter for more cancer news: @MedscapeOnc.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: