Clinical and Health Economic Outcomes of Alternative HER2 Test Strategies for Guiding Adjuvant Trastuzumab Therapy

James A Lee; Megan Shaheen; Thomas Walke; Matt Daly

Disclosures

Expert Rev Pharmacoeconomics Outcomes Res. 2011;11(3):325-341. 

In This Article

Results

The results are presented following the same four subsections as presented in the methods section: literature, concordance analysis, baseline health economic and outcomes analysis, and economic analysis results under alternative assumptions. All economic results are presented as incremental cost–effectiveness ratios (cost per QALY) from a US national Medicare payer perspective using 2009 national reimbursements.

Literature

Our analysis of IHC and FISH includes 46 studies, of which 26 were drawn from the Dendukuri et al.[10] and Caudros and Villegas[22] papers. Our search identified seven publications prior to 2006 but not included in Dendukuri et al. or Cuadros and Villegas, plus four additional studies published after 2006. We also analyzed the results by method of IHC separately for publications where multiple methods were used and test performance was presented separately. As previously stated, in order to understand the potential impact of carrying out a meta-analysis or pooled-analysis versus individual analyses, we included Dendukuri et al. and Cuadros and Villegas pooled test performance data as two distinct analyses.

Regarding gene-based assays, seven out of the 21 studies identified comparing CISH and FISH were post-2006 and had a substantial mix of methods: 11 used a single probe only, five used a dual probe only, four used a mix of single and dual probe, and one used a triple probe. All five studies comparing SISH and FISH were post-2006 and all used dual-probe SISH.

Note that one study by Gong et al., which showed 95.0, 96.2 and 97.5% correlation between CISH and FISH using single-probe CISH for three pathologists, respectively, and 91% concordance between the three pathologists, was excluded from this analysis as the results did not separately present positive and negative concordance to support clinical and economic analysis.[28]

Concordance Analysis

Table 2 and Table 3 present the results of the concordance analysis for IHC versus FISH and the three gene-based assays, respectively. The results comparing SISH and FISH are presented in the last five rows of Table 3. A few columns are explained below.

IHC 0/1+ When FISH (Negative Concordance) Negative concordance was defined in the ASCO/CAP guidelines as the proportion of samples with a negative result for the assay under consideration (IHC) relative to the samples with a negative result for reference assay (FISH). Table 3 provides analogous results for CISH and SISH relative to FISH.

IHC 3+ When FISH+ (Positive Concordance) Positive concordance was defined in the ASCO/CAP guidelines as the proportion of samples with a positive result for the assay under consideration (IHC) relative to the samples with a positive result for reference assay (FISH). Table 3 provides analogous results for CISH and SISH relative to FISH.

Pass ASCO/CAP Guidelines Both the negative and positive concordance between IHC and FISH, as the reference assay, after excluding equivocal IHC results, are greater than 95%. Table 3 provides analogous results for CISH and SISH relative to FISH, but equivocal results are not excluded as these are seldom specifically identified in the publications.

Confirm IHC 2+ (Specificity) Agreement between IHC and FISH where IHC equivocal results (2+) are confirmed and therefore concordant with FISH. This is the equivalent to the specificity of the confirm IHC 2+ test strategy assuming FISH as the reference assay.

Confirm IHC 2+ (Sensitivity)

Agreement between IHC and FISH where IHC equivocal results (2+) are confirmed and therefore concordant with FISH. This is equivalent to the sensitivity of the confirm IHC 2+ test strategy assuming FISH as the reference assay.

Key Observations From the Concordance AnalysisIHC & FISH Only seven of the 46 studies comparing IHC and FISH demonstrate both positive and negative concordance that meet the ASCO/CAP guideline of 95% or greater. Neither of the pooled analyses performed by Dendukuri et al.[10] or Cuadros and Villegas[22] presented performance results that exceeded the ASCO/CAP guidelines. Thus, if IHC were introduced as a new technology, it would relatively infrequently meet ASCO/CAP concordance guidelines if FISH were used as the reference assay. To understand the concordance rates in terms of false-negatives and -positives, Table 4 presents the number of the 46 studies with a false-negative (IHC 0/1+ and FISH+) and false-positive (IHC 3+ and FISH) error rate within a given range. The error rate is expressed as a number of discordant results per 1000 tests performed. The results show that two-thirds of the 46 studies have false-positive and -negative error rates greater than ten per 1000 tests performed using a confirm IHC 2+ protocol.

CISH & FISH Of 21 studies comparing CISH and FISH (one publication by Gong et al. had two distinct study sites and results[28] so they are presented here as two distinct analyses), 14 met ASCO/CAP concordance guidelines. Of note regarding these 14 studies: the one that used triple-probe analysis passed the guidelines; all five that used dual-probe analysis on all samples also met or surpassed the guidelines; and three of four that used dual-probe analysis on some cases met or surpassed the guidelines. On the other hand, only five of 11 studies that used single-probe CISH only met the concordance guidelines when FISH served as the reference assay. Single-probe CISH has poorer concordance with FISH relative to testing strategies that include dual-probe CISH.

SISH & FISH Of five studies comparing SISH and FISH, three have perfect concordance and therefore surpass the ASCO/CAP guidelines. However, two studies (Dietal et al.[29] and Shousha et al.[30]), have positive concordance values of 81.8 and 84.6%, well below the guideline threshold of 95%.

With a few exceptions, sufficient concordance between IHC and FISH is not observed to meet or surpass ASCO/CAP guidelines, while concordance between dual-probe CISH, SISH and FISH is high and quite frequently surpasses ASCO/CAP guidelines for both 95% positive and negative concordance.

Baseline Health Economics & Outcome Results

Table 2 and Table 3 also present the baseline health economics and outcome results comparing IHC and FISH and the three gene-based assays, respectively. Example cost and QALY calculations are shown in the Appendix. The definitions of each of the columns that present economic and outcome results are as follows.

Cost–effectiveness of Primary FISH versus Confirm IHC 2+

The cost–effectiveness of primary FISH versus confirm IHC 2+ presents the incremental cost per QALY to adopt a primary FISH strategy over a confirm IHC 2+ strategy. When a strategy is deemed 'dominant', it means that the strategy is observed to lower costs and improve outcomes as measured in QALYs (e.g., primary FISH dominant means that a primary FISH strategy is likely to both reduce healthcare costs and improve health outcomes). A positive value, such as US$24,421, as shown for the Ainsworth study, means that one expects to incrementally pay US$24,421 per QALY using the primary FISH strategy relative to a confirm IHC 2+ strategy.

Cost–Effectiveness of Primary FISH versus Confirm IHC 2+ & 3+ The cost–effectiveness of primary FISH versus confirm IHC 2+ and 3+ presents the incremental cost per QALY to adopt a primary FISH strategy over a confirm IHC 2+ and 3+ strategy. The latter, as previously noted, is used in Belgium and Finland and recommended in Dendukuri et al.'s analysis.

Cost–Effectiveness of Confirm IHC 2+ & 3+ versus Confirm IHC 2+ The cost–effectiveness of confirm IHC 2+ and 3+ versus confirm IHC 2+ presents the incremental cost per QALY to adopt a confirm both IHC 2+ and 3+ strategy over a confirm IHC 2+-only strategy.

Cost–Effectiveness of FISH versus CISH/SISH (Dual Probe) The cost–effectiveness of FISH versus CISH/SISH (dual probe) presents the incremental cost per QALY to adopt a primary CISH or SISH strategy versus a primary FISH strategy. In these cases, either dual-probe CISH was used, with one exception where Vera-Román et al. used a three-probe CISH test.[31] All SISH tests were dual probe.

Cost–Effectiveness of FISH versus CISH (Single Probe) The cost–effectiveness of FISH versus CISH (single probe) presents the incremental cost per QALY to adopt a primary CISH versus a primary FISH strategy. In these cases, either only single-probe CISH was used or, where identified, a strategy of using primarily a single-probe CISH assay with a second probe used only on a subset of samples.

The results of the baseline economic analysis by test strategy were as follows.

Cost–Effectiveness of Primary FISH versus Confirm IHC 2+ The primary FISH strategy was dominant (lower cost, higher QALYs) in 22 observations, and cost effective in 22 additional observations, with a range of incremental costs per QALY of US$104–39,162. Two observations, Lebeau with 78 test samples[32] and Vera-Román with 50 test samples,[31] had perfect concordance between IHC and FISH so the confirm IHC 2+ strategy was dominant. Overall, with 44 out of 46 studies resulting in primary FISH being a cost-saving or cost-effective strategy (below US$50,000 per QALY), it appears that primary FISH is a preferred strategy over confirmation of IHC 2+.

Cost–Effectiveness of Primary FISH versus Confirm IHC 2+ & 3+ The cost–effectiveness of primary FISH versus confirm IHC 2+ and 3+ presents the incremental cost per QALY to adopt a primary FISH strategy over a confirm IHC 2+ and 3+ strategy. Three observations have primary FISH dominant and seven observations have confirm IHC 2+ and 3+ dominant. All of the latter have 100% positive concordance (IHC 3+ and FISH+). A total of 34 observations resulted in primary FISH being a cost-effective strategy relative to confirm IHC 2+ and 3+, with a range of incremental cost per QALY of US$21,889–49,600. Two studies, using automated cellular imaging system, provide only IHC-positive and -negative results and were excluded from this analysis as IHC 2+ results were not separately identified. Thus, 37 out of 44 observations result in primary FISH being a preferred strategy over confirm IHC 2+ and 3+ from a health economic and outcomes perspective using a threshold of US$50,000 per QALY. Essentially, a 100% positive concordance is required for confirm IHC 2+ and 3+ to be a dominant strategy over primary FISH.

Cost–Effectiveness of Confirm IHC 2+ & 3+ versus Confirm IHC 2+ Of 44 studies, 36 result in confirm IHC 2+ and 3+ dominating the confirm IHC 2+ strategy. The remaining eight studies, where the observed negative concordance between IHC and FISH (IHC 0 or 1+, FISH) is greater than 99.8%, resulting in confirm IHC 2+ being the dominant strategy. Thus, a very high negative concordance is required for confirm IHC 2+ to be a preferred strategy over confirm IHC 2+ and 3+.

Cost–Effectiveness FISH versus CISH (Dual Probe) For dual-probe CISH versus FISH, two observations resulted in CISH and FISH being equal, three resulted in FISH being a dominant strategy and one resulted in FISH being cost effective at an incremental cost of US$23,258 per QALY. Since US Medicare reimbursement rates are on a per-probe basis making the test reimbursement equal, these results are sensitive to small differences in concordance.

Cost–Effectiveness of FISH versus CISH (Single Probe) Of the 15 studies where CISH is used as a single-probe assay in some or all samples, six observations result in FISH being dominant and five result in FISH having an incremental cost–effectiveness per QALY ranging from US$11,033 to US$38,916. There were four observations where CISH is dominant and all have 100% concordance between FISH and CISH.

Cost–Effectiveness of FISH versus SISH The cost–effectiveness results show that FISH is dominant in four out of the five observations and cost effective at an incremental cost of US$11,632 per QALY in one observation. As with dual-probe CISH, since FISH and SISH test reimbursement is equal, these results are sensitive to small changes in test performance.

Overall, the gene-based assays, in particular when comparing FISH and dual-probe CISH, have high concordance and an uncertain result regarding relative cost–effectiveness. Primary FISH, however, is nearly always cost effective or dominant relative to strategies using confirmation of IHC 2+ or IHC 2+ and 3+. Given the relatively high cost of AT therapy compared with HER2 test costs, a strategy of confirm IHC 2+ and IHC 3+ appears preferable to a strategy of confirm IHC 2+ only.

Economic Analysis Results Under Alternative Assumptions

Given the challenges of performing a pooled or meta-analysis, especially with substantial variation in the methods used to perform IHC and CISH, we evaluated the economic analysis results under alternative assumptions of test performance, AT therapy cost and AT therapy benefit in terms of QALYs. The analysis was performed using the previously described Monte Carlo simulation applied separately to seven actual studies considered in the baseline economic analysis.

Table 5 presents, for this select subset of studies, both the mean cost–effectiveness value and the probability that the cost–effectiveness is less than a threshold of US$50,000 per QALY. The subset includes studies that passed the ASCO/CAP guidelines but did not have perfect concordance between IHC and FISH, the pooled analyses of two systematic reviews (Dendukuri et al.[10] and Cuadros and Villegas[22]) and one frequently referenced study (Press et al.[14]) comparing local laboratory-performed Dako-HercepTest™ with central laboratory-performed FISH.

Examining the Dowsett et al. study results more closely, even though the concordance between IHC and FISH is greater than 98%, one can be nearly certain (99.9%) that primary FISH is a cost-effective strategy relative to confirmation of IHC 2+ results for the test performance observed in this study.[33] Regarding the confirm IHC 2+ and 3+ strategy, in more than 56% of the simulation runs, primary FISH had a cost–effectiveness ratio of less than US$50,000 and had an average cost–effectiveness ratio of US$50,968 per QALY. The remaining results confirm over a wide range of assumptions that the primary FISH protocol is typically cost effective relative to protocols based on confirmation of IHC results.

The high rate of concordance between dual-probe CISH, SISH and FISH does not provide as clear a conclusion with respect to health economic outcomes. Simulation analysis results, summarized in Table 6, show the scenarios where a 95% or greater proportion of model runs resulted in the reference assay having a cost per QALY less than US$50,000 relative to an alternative assay.

The results show that for price differentials less than US$400, for example, an alternative assay must have a sensitivity and specificity, relative to the reference assay, equal to or exceeding 97 and 98%, respectively, to be considered the more cost-effective assay with high confidence. Lower price differentials require even better performance of the alternative assay and price differentials less than US$100 require near-perfect concordance. The results are not sensitive to varying levels of HER2 overexpression in the population, as shown for a US$200 price differential and rates of 15, 20 and 25% HER2 overexpression. Thus, correlation with therapy response is key to determining the most clinically sound and cost-effective assay, rather than assay price, even for assays with a US$400 price differential.

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.

processing....