Waterfall Plots Oversell Response Rates in Cancer Trials

By Reuters Staff

May 30, 2019

NEW YORK (Reuters Health) - Graphs known as waterfall plots exaggerate cancer-treatment response rates, and their use in oncology literature is increasing, according to new findings.

"Waterfall plots provide a visual overestimate of response rate of cancer therapies and should be evaluated with caution," Dr. Myung Sun Kim and Dr. Vinay Prasad of Oregon Health and Science University in Portland conclude in JAMA Network Open, online May 17.

Waterfall plots are histograms ordered based on the best percentage change in tumor size, with each vertical column representing a single patient. Positive values show increases in tumor size and negative values indicate tumor shrinkage.

"Prior research has suggested that waterfall plots may be subject to interreader variation, with variability in the final plot based on the particular scorer or reader of tumor measurements," the authors note.

Dr. Kim and Dr. Prasad evaluated the correlation between treatment response represented by waterfall plot and reported as study outcomes in 126 studies in six top oncology journals, most of which were phase 1 or phase 2 clinical trials.

Ninety-seven included overall response rate based on investigator assessment, whereas overall response rate was assessed centrally in 42.

Visual response rates represented in waterfall plots were 6.1% higher than rates based on review by investigators and 12% higher than response rate based on central review, both significant findings.

While waterfall plots were not included in any original articles sampled from 2004, 7% of articles published in 2018 included waterfall plots.

"Considering that response rate is considered a key outcome in reporting efficacy of novel therapies, it is concerning that waterfall plots represent an overstatement of results in many cases," Dr. Kim and Dr. Prasad write. "With an increasingly large number of therapies competing for attention and resources, the perception of clinically significant antitumor activity is often critical in securing future funding and approval."

They conclude: "To maintain the utility of waterfall plots while preserving the integrity of reporting outcomes of clinical trials, we suggest clear statements about dropout rates and reasons for dropout. Waterfall plots may evolve to include missing data points to avoid misrepresentation of response rates. In addition, we suggest use of landmark plots and second-best scans, as well as a clear statement about the RECIST 1.1 response rate."

SOURCE: https://bit.ly/2Mf0XG5

JAMA Netw Open 2019.