Use of Number Needed to Treat in Cost-effectiveness Analyses

Vishvas Garg MBA BPharm, PhD; Xian Shen MS, PhD; Yan Cheng MS, PhD; James J Nawarskas PharmD BCPS; Dennis W Raisch PhD MS RPh

Disclosures

The Annals of Pharmacotherapy. 2013;47(3):380-387. 

In This Article

Discussion

The number of CEAs published is increasing each year. The Health Economic Evaluations Database (HEED) alone consists of more than 44,000 health economic evaluations and is updated monthly with approximately 250 new studies.[88] We found only 62 health economic evaluations that incorporated NNT. Therefore, the studies incorporating NNT into CEA represent a small fraction of published health economic evaluation studies.

We classified approaches to incorporate NNT (or NNH) into CEAs into 2 categories: (1) indirect methods, and (2) direct methods. Indirect methods involve reporting NNT (or NNH) separate from QALYs or other health economic outcome measures, such as CEA ratios or incremental CEA ratios. All CEAs that currently incorporate NNT/NNH fall under this category, based on our review. For example, a CEA published in the Annals of Internal Medicine (2010 impact factor = 16.729) compared the cost-effectiveness of implantable cardioverter-defibrillator and amiodarone after myocardial infarction and reported NNT (defined as NNT to prevent 1 arrhythmic death) separately from cost per QALY gained as follows: in comparison with amiodarone, the ICER for implantable cardioverter-defibrillator was $71,700–557,900 per QALY gained. NNT was 9.6 to 21.2 for implantable cardioverter-defibrillator and 14.4–146.8 for amiodarone, depending on the ejection fraction (varied from <0.3 to >0.4).23 In another example, a CEA published in British Medical Journal (2010 impact factor = 13.471) examined the cost-effectiveness of community occupational therapy for older patients with dementia, reporting NNT per 1 successful treatment of dementia as 2.8 (95% CI 2.7–2.9). Cost per QALY gained for occupational therapy, in comparison with usual care, was $10,666/QALY.[28] These 2 articles demonstrate a concise manner for interpreting NNT alongside CEA.

Direct methods involve using mathematical models to establish direct functional relationships between NNT (or NNH) and health economic outcome measures. Such relationships vary on a case-by-case basis and require adjustment for other factors, such as the time horizon of the analysis. For example, Kristiansen and Gyrd-Hansen found that, as a treatment's effectiveness changes over time (improves or decreases), NNT also varies and the relationship between NNT and quality of life changes.[89] As a result, use of NNT ratios alone in long-term CEA studies may be misleading.

Examples of such direct methods regarding risk-benefit assessment can be found in the literature.[90] These include outcome measures such as Quality-adjusted Time Without Symptoms and Toxicity (Q-TWiST), Relative Value Adjusted Number Needed to Treat (RV-NNT) and RV-NNH, and Incremental Net Health Benefit (INHB).[90–92] For instance, Q-TWiST is an estimator to determine QALYs by adjusting life expectancy with the respective utility weights by summing (1) QALYs based on time with toxicity from chemotherapy, (2) followed QALYs for time free of symptoms of disease or chemotherapy toxicity, (3) plus QALYs for time with disease symptoms.[92] As proven, Q-TWiST is an asymptomatically consistent and normally distributed estimator with variance that can be consistently calculated.[93] The establishment of a direct functional relationship between NNT and QALY is beyond the scope of this review. Nonetheless, further research in this field may lead to better understanding of such relationships.

Indirect methods to incorporate NNT (or NNH) into CEAs may be more transparent than direct methods, since the former do not suffer from constraints of establishing a functional relationship between NNT (or NNH) and health economic outcomes measures. Either way, QALYs are health economic measures incorporating both benefits and harms of each treatment's outcome.[17] In contrast, NNT (or NNH) simply summarizes the proportion of patients impacted positively (or negatively) by the treatment. Further, QALY values at the terminal ends of decision tree models summarize the impact of all outcomes along the branch of the decision tree on health-related quality of life, while NNT (or NNH) usually describes 1 or 2 potential outcomes. NNT cannot be compared across different health conditions and response variables (eg, interventions to treat the common cold vs cancer). Therefore, QALYs provide significantly more information regarding the impact of different outcomes that may result from therapeutic alternatives rather than the few outcomes addressed by NNT or NNH. Incorporating both values may help clinicians perceive the relationships between the measures.

Notably, we found that 40.5% of CEA articles incorporating NNT involved comparisons of a specific disease treatment and an additional 40.5% involved disease prevention. Only a small proportion of studies involved disease management (14.5%) or educational interventions (4.3%). This reflects the relative proportions of treatment and prevention CEAs versus disease management CEAs. It may also be associated with the multiple outcomes measured by disease management studies, which would require the reporting of several NNT values.

We found that 72.4% of all of the eligible articles were published in the top quartile of journals by impact factor and only 2% of the articles were published in the bottom quartile (impact factor ≤1). Since most articles that integrate NNT into CEAs were published in clinician-practice-focused journals (78.3%), these findings indicate that CEA studies incorporating NNT are generally of high clinical impact. This does not indicate that CEA studies are of higher quality, which would involve assessment of the methodologic sophistication and scientific rigor in the conduct of the study, among other factors. The data sources for the CEA studies incorporating NNT were based on published work (53.4%), followed by studies reporting data from randomized clinical trials (24.6%, Table 1). This may simply reflect the use of literature review in high proportions of published CEAs. NNTs have utility only when the evidence on which they are based fulfills criteria of quality, validity, and size.[94] Of all types of studies, randomized clinical trials, especially large randomized trials, are most likely to best fulfill all of these criteria. Nonetheless, we found only 24.6% of randomized trials (O-RCT-CEA plus RCT-CEA) that incorporated NNT.

Further research regarding the co-reporting of NNT within CEAs might involve (1) developing surveys of clinicians comparing their understanding of CEAs with or without NNTs, (2) establishing tools for adjusting NNT/NNH by quality of life measures, similar to Q-TWiST, RV-NNT, RV-NNH, or INHB, (3) comparing of implementation of CEAs that include NNT versus those that do not, or (4) incorporating other risk-benefit assessment techniques, such as Multi-Criteria Decision Analysis, into estimates of NNT/NNH.[90]

Our study is not without limitations. First, we excluded non-English-language literature from our search. Although not likely to impact our findings substantially, 8 additional studies in other languages came up in our search before the application of inclusion/exclusion criteria. Second, we did not consider the time horizon of each study as an evaluation criterion in our study. As previously shown, NNT should not be used for measures of benefit in CEAs when benefits or adverse effects vary over time.[89] Third, some CEA studies incorporating NNT may not have been published because of the existing bias in the literature (ie, only CEA studies having favorable or positive results were published). Fourth, because of resource constraints, we searched only 2 databases (PubMed and EconLit). Review of other databases, such as EMBASE, may reveal additional publications. However, per our previous experience, we expected to find few unique eligible articles by searching EMBASE.

We found a relatively small number of studies incorporating NNT into the results of CEA. The majority of the studies were published in clinician-practice-focused journals. While there are limitations to NNT, these results are commonly used clinically and can serve as a bridge to help clinicians relate these results to more detailed, and perhaps more useful, CEA. This would involve including NNT information in CEA, which may help bridge that gap of understanding and assist decision-makers in applying CEA results to decisions in clinical settings. Further research may be needed to determine whether these studies are more understandable by clinicians and increase implementation of CEA results in clinical settings. Additionally, researchers may need to explore possibilities as to how best to incorporate theoretically sound NNT/NNH values into health economic evaluations.

processing....