Absolute Measures of Efficacy, Harms Still Missing From Most Clinical Trial Reports

By Frederik Joelving

November 25, 2020

NEW YORK (Reuters Health) - Absolute measures of efficacy and harms are usually missing from reports of clinical trials in high-impact medical journals, flying in the face of long-standing guideline recommendations, researchers have found.

Relative risk measures - say, a 50% reduction - can be misleading, because they don't take into account the frequency of a given outcome of interest - say, 0.5% in one study arm and 1% in the other. As a result, they may exaggerate differences of little clinical importance.

"When absolute risks are presented for each intervention, and are expressed in terms of whole numbers, they can improve interpretability," said Dr. Joshua D. Wallach of the Yale School of Public Health, in New Haven, Connecticut.

"Our study suggests that opportunities remain to further improve the reporting of absolute measures in controlled clinical trials," he told Reuters Health by email. "In particular, it may be necessary for journals and reporting guidelines/checklists to include more stringent requirements, to help ensure the interpretability of trial results."

Since 2010, the Consolidated Standards of Reporting Trials (CONSORT) statement has recommended that "both relative and absolute effect sizes" should be used when presenting binary outcomes (https://bit.ly/33bYgv6).

To assess the state of affairs in trial reporting, Dr. Wallach and his colleagues identified the six most-cited medical journals in 2019 and reviewed all issues published in 2001, 2007, 2013 and 2019. They looked for controlled clinical trials that tested if an intervention was superior to a control condition and reported abstract-level binary outcomes.

Among 875 trial reports published in The New England Journal of Medicine, The Lancet, JAMA, The BMJ, JAMA Internal Medicine, or Annals of Internal Medicine, 29% included at least one estimate of absolute risk reduction (ARR).

Nine percent reported one or more estimates of number needed to treat (NNT). And just 0.9% reported one or more estimates of number needed to harm (NNH), the safety equivalent of NNT.

"We did observe low levels of NNH reporting, which supports previous concerns that harms are too often under-reported in research," Dr. Wallach said.

A third of the trial reports included at least one estimate of ARR, NNT or NNH, the researchers report in JAMA Internal Medicine. ARR reporting rose significantly over the study period, from 19% to 37%, but NNT/NNH reporting remained stable around 9%.

"It was surprising to find that despite calls for increased reporting of NNT/NNH over the past decade, reporting of these measures in controlled trials remained constant between 2001 and 2019," Dr. Wallach told Reuters Health.

Relative and absolute measures both have strengths and weaknesses, he noted. NNT may not be easily applicable to patients outside the study population, for instance.

He suggested that "guidelines should clarify when ARR, NNT, and/or NNH measures should be reported, along with examples and links describing how to calculate, interpret, and report these measures along with their 95% confidence intervals."

SOURCE: https://bit.ly/33cNvZi JAMA Internal Medicine, online November 23, 2020.