Should cancer drugs be evaluated by regulators with a measure that is related to the magnitude of clinical benefit — and not just a statistical advantage? And would such a new standard for "significance," in turn, improve cancer care?
These questions and others are raised by a new study that reviewed 277 randomized clinical trials (RCTs) of systemic therapies for 4 different solid tumor types that took place in the past 5 years (2011–2015).
The tumor types were breast cancer, non-small cell lung cancer, colorectal cancer, and pancreatic cancer.
The investigators found that the experimental therapy was statistically superior to the control arm in 138 (50%) of the 277 trials.
But were these results clinically meaningful?
To get a sense of that, the researchers inserted the results into a new tool, the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). The scale was recently released with the purpose of providing scores that indicate "meaningful clinical benefit," as reported by Medscape Medical News. (The scale can be reviewed here.)
Among the 138 RCTs in the study that showed a statistically significant outcome, only 43 (31%) met the ESMO-MCBS clinical benefit threshold, report the authors, led by Christopher Booth, MD, an oncologist and researcher at Queen's University in Kingston, Ontario, Canada.
Thus, overall, when considering all of the RCTs in the study, only 15% (43/277) met a threshold for meaningful clinical benefit in their results.
"I think the most important findings from our work is that only 15% of all contemporary randomized controlled trials of anticancer therapies [in the 4 tumor types] are identifying new treatments that offer meaningful benefit to patients," Dr Booth told Medscape Medical News.
The study was published online October 13 in the Annals of Oncology.
An oncologist not involved with the study believes that the field of oncology should aim high.
"We must work to improve our clinical trials and drugs, and aspire for meaningful not marginal benefits. Our patients deserve no less," said Vinay Prasad, MD, MPH, a medical oncologist at Oregon Health and Sciences University in Portland, who was asked for comment.
"I think that most reasonable people would say cancer drugs that extend survival a month or two, often at the price of significant toxicity and cost, do not offer clinically meaningful benefit," he added.
A significant part of the current problem is how RCTs are designed, explain the authors.
Less than a third of the studies in the trial (31% [70/226]) were designed to detect an "effect size" that could meet ESMO-MCBS thresholds, they report.
Effect size is the magnitude of difference between groups in a study. A P value, on the other hand, can show whether an effect exists but does not reveal the size of the effect.
There also is an arbitrariness to P values. "Are a P = .049 and a P = .052 really any different?" Dr Booth asked.
"Statistical significance does not convey the extent to which patients might benefit from cancer therapies," he summarized.
In the new study, the RCTs, 83% of which were sponsored by industry, were mostly not aiming for large effect sizes, suggest the authors.
"Our data demonstrate that the majority of contemporary 'mega-trials' are designed to detect a very small effect size that in most cases would not be considered 'clinically significant'," they write.
The trials keep an eye on the prize of regulators' approval, they suggest.
"It is likely that the design of contemporary RCTs is at least in part driven by 'thresholds' inherent in the regulatory approval of a novel therapy," the authors say.
That does not mean RCTs can't show big benefits, say the authors: "Despite being powered to detect a small benefit, large trials may still meet meaningful clinical benefit by detecting a larger effect size than anticipated."
RCTs keep getting bigger and bigger in terms of patient numbers, point out the authors.
In the new study, the median number of patients in the trials was 532.
But the authors note that sample size has been increasingly steadily in recent decades. In another of their studies (Ann Oncol. 2012;23:1646-1651), they reported that the median sample size in cancer drug trials was shown to have increased from 100 in the 1970–1980s, to 446 in the 1990s, to 722 in 2005–2009.
There was another trend identified in that earlier study: Author interpretation of "benefit" has evolved over time.
"Despite effect size remaining stable over time, authors of modern RCTs were substantially more likely to strongly endorse the experimental arm of the trial as the new standard of care," the study authors write.
ESMO is not the only major cancer group seeking to promote value in cancer drug reviews.
The American Society of Clinical Oncology (ASCO) has also proposed a method for evaluating clinical value with a threshold for meaningful clinical benefit, the ASCO Value Framework (ASCO-VF), as reported by Medscape Medical News.
The intent of the two frameworks differ, say the authors.
They say that the ESMO-MCBS was derived to "frame the appropriate use of limited public and personal resources to deliver cost-effective and affordable cancer care." Differently, the ASCO-VF was designed to "assist in facilitating shared decision making with patients about clinical benefits and costs."
"Recent guidelines by ASCO and ESMO aim to take a stab at some consensus [for clinically meaningful benefit], just like we did for P values decades ago. I applaud this effort," observed Dr Prasad.
Dr Booth says the ASCO and ESMO efforts and their respective clinical benefit thresholds are a "starting point for the conversation" about this approach.
What if drugs were evaluated and approved by regulators based on a clinical value measure and not a statistical value? How might the practice of oncology be affected?
Both Dr Prasad and Dr Booth took a turn imaging that scenario.
"Not only would practice be different if we demanded drugs to be clinically significant, but drug development would be turned on its head," answered Dr Prasad.
"Industry would no longer be incentivized to conduct overpowered trials to find statistically significant, but clinically marginal benefits. Instead, we might see fewer trials, but the drugs being tested would be ones that people thought were more likely to improve outcomes meaningfully," he added.
The system might yield fewer drug approvals, but "there would be a lot fewer Avastin trials, for instance," he said, citing an extensively investigated drug that has produced marginal benefits in many cancer trials.
"We could focus on the best drugs in the pipeline," said Dr Prasad. "We would still get imatinib, Rituxan, and Herceptin."
Dr Booth was more generalized in his vision.
"I think the optimal system would incorporate treatments that have real clinical benefit in endpoints that matter to patients (ie, overall survival or quality of life) and have treatments that are based on well-designed clinical trials," he said.
Magnitude of benefit must be included in the evaluation of cancer drugs, Dr Booth concluded.
"Oncologists, policymakers, and payors need to carefully consider the magnitude of benefit associated with new cancer therapies before simply adopting a new expensive and potentially toxic agent that might offer little real benefit to patients," he said.
Dr Booth and Dr Prasad have disclosed no relevant financial relationships. One study coauthor was a member of the ESMO task force that designed the ESMO-MCBS.
Ann Oncol. Published online October 13, 2016. Abstract
Follow Medscape senior journalist Nick Mulcahy on Twitter: @MulcahyNick
Follow Medscape Oncology on Twitter: @MedscapeOnc
Medscape Medical News © 2016 WebMD, LLC
Send comments and news tips to news@medscape.net.
Cite this: Judging Cancer Drugs by Clinical, Not Statistical, Measures - Medscape - Nov 04, 2016.
Comments