A Standardised, Generic, Validated Approach to Stratify the Magnitude of Clinical Benefit That can be Anticipated From Anti-cancer Therapies

The European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS)

N. I. Cherny; R. Sullivan; U. Dafni; J. M. Kerst; A. Sobrero; C. Zielinski; E. G. E. de Vries; M. J. Piccart


Ann Oncol. 2015;26(8):1547-1573. 

In This Article


Inherent Challenges in Developing Standard Clinical Benefit Scale

The substantial variability of study designs (crossover, non-crossover and partial crossover), planned outcomes and reported outcomes inherently challenge the process of developing a unified scale of clinical benefit. This challenge is all the greater in an era in which both researchers and regulatory authorities are employing surrogate outcome indicators as primary end points for both research and registration criteria.[5] A unified scaling approach requires a process of relative weighting of evidence that demands conceptual rigor, careful reviews of the validity and strength of surrogate end points and clinical nuance.

Validity of the ESMO-MCBS

The ESMO-MCBS version 1 (ESMO-MCBS v1.0) provides an objective and reproducible approach that allows comparisons of the magnitude of benefit between studies that incorporate different primary outcomes (OS, PFS, QoL) and different designs through a process of variable weighting of primary outcomes and adjustments for significant secondary outcomes and toxicity.

The development process has been compliant with the criteria for 'accountability for reasonableness' which represent the ethical gold-standard for a fair priority setting process in public policy.[134,135] The validity of the ESMO-MCBS is derived from (i) clinically relevant and reasonable criteria for prioritisation of different types of benefit, i.e. that cure takes precedence over deferral of death, direct end points such as survival and QoL take precedence over less reliable surrogates such as PFS or RR and that the interpretation of the evidence for benefit derived from indirect primary outcomes (such as PFS or RR) may be influenced by secondary outcome data, (ii) coherence: procedural agreements regarding the evidence to be used/not used, how it will be analysed and evaluated, and precautions to minimising bias (including conflict of interest issues) based upon an understanding of the relative strengths and weaknesses of the usual measured outcomes OS and QoL, and their surrogates[13–23,136] and rigorous bio statistical review, (iii) wide applicability over a range of solid cancers and a range of prognoses that have been rigorously tested (iv) statistical validity and (v) a transparent process of development with scope for peer review, appeal and revision.

ESMO-MCBS scores for a specific therapy are not generalisable to indications outside the confines of the context in which they have been evaluated. Consequently, the ESMO-MCBS score for a particular medication or therapeutic approach may vary depending on the specifics of the indication and may vary between studies.

Limitations of the ESMO-MCBS v1.0

The ESMO-MCBS can only be applied to comparative research outcomes; it is therefore not applicable when evidence of benefit derives from single-arm studies. This limits its utility in the uncommon situation in which registration is granted on the basis of outcomes reported from single-arm studies.

The process of relative weighting of evidence and the thresholds for HR and absolute gains involves judgements and subjective considerations which are amenable to dispute and challenge and indeed, this is invited as part of the dynamic process of peer review and further development.

Factors That may Skew or Alter ESMO-MCBS Scores

Control Arm Evaluation. The ESMO-MCBS evaluates data derived from comparative research, either randomised phase II[137] or phase III studies or cohort studies. The validity of the results may be influenced by the quality and design of the study. Design issues are critical insofar as studies that incorporate a relatively weak control arm may generate the impression of exaggerated benefit. This was manifest in studies evaluating treatment options for hormone refractory prostate cancer where one study used mitoxantrone/prednisone as the control arm[73] based on the findings of a phase III study comparing prednisone versus the combination of prednisone and mitoxantrone which demonstrated improved QoL but no survival advantage for the combination therapy[138] and others used prednisone alone[69] or placebo.[70]

Crossover. Crossover, or subsequent treatment of control arm patients with biologically similar agent, severely compromises the ability to derive reliable data regarding the survival advantage of treatments in phase III studies. This factor may impact on OS results as illustrated by the study of dacarbazine versus ipilimumab in metastatic melanoma[126] in which the evidence for survival advantage was diluted by the crossover provision in the study. In some instances in which strong PFS advantage is seen, crossover of this type will obscure the potential survival benefit of the new treatment. Statistical approaches to estimate longer term clinical outcomes despite substantial treatment crossover have been developed,[139,140] and applied.[141–144] While these approaches are encouraging, they incorporate a range of assumptions and are not universally accepted.[145]

Unbalanced Crossover. In other instances, unbalanced crossover may exaggerate differences in survival. For instance, in the PEAK study comparing FOLFOX6 with either bevacizumab or panitumumab among the patients with KRAS wild-type tumours, only 38% of those in the bevacizumab arm received any EGFR antibody in subsequent therapy.[146] Although this study showed a survival advantage of 9.9 months over a baseline of 24.3 months for patient initiated on treatment with panitumumab, it remains unclear as to whether this was affected by the sequence of treatments or if it was the result that more than half of the patients in the bevacizumab arm were never exposed to an EGFR antibody.

Follow-up Reports. In some studies, first reports are followed up with subsequent further relevant data analysis. This is particularly true when mature survival data were not available in studies with a primary outcome of PFS or DFS and in studies that have incorporated post hoc stratification based on refined appreciation of tumour biology that may impact on outcome evaluation.

Both of these phenomena were observed in the three publications reporting the findings from the same study on FOLFOX4 ± panitumumab for the first-line treatment of KRAS wild-type colorectal cancer.[74,78,79] The study, which did allow for crossover to other EGFR antibodies, had PFS as a primary end point. The initial publication demonstrated a modest PFS advantage with non-significant median OS gain.[78] The subsequent publication of mature data demonstrated a significant OS gain[79] with the greatest benefit restricted to patients with KRAS, NRAS, BRAF wild-type tumours.[74] Almost identical data maturation was observed in the CRYSTAL study evaluating FOLFIRI± cetuxumab in the same clinical setting.[76,80,81]

Maturation of survival data also increased the ESMO-MCBS score of vemurafenib in the treatment of metastatic melanoma[116,117] from ESMO-MCBS 3 based on PFS to 4, based on OS.

Using Data From the ESMO-MCBS

The ESMO-MCBS incorporates a structured, rational and valid approach to data interpretation and analysis that reduces the tendency to have judgements affected by bias or uninformed and/or idiosyncratic data interpretation. Consequently, application of the scale reduces the likelihood that statements of clinical benefit will be distorted by either overestimation or overstatement on one extreme or, nihilism at the other. This structured and disciplined approach to deriving estimates of clinically meaningful benefit from published data can be used in a range of settings.

Public Policy Applications. Grading derived from the ESMO-MCBS provides a backbone for value evaluations for cancer medicines. Medicines and therapies that fall into the ESMO-MCBS A + B for curative therapies and 4 + 5 for non-curative therapies should be emphasized for accelerated assessment of value and cost-effectiveness. While a high ESMO-MCBS score does not automatically imply high value (that depends on the price), the scale can be utilised by to frame such considerations[147] and can help public policy-makers advance 'accountability for reasonableness' in resource allocation deliberations.[134,135]

Formulation of Clinical Guidelines. The prevailing current practice of the National Comprehensive Cancer Network (NCCN), American Society of Clinical Oncology (ASCO), ESMO and the National Cancer Institute (NCI) in their guidelines is to grade the 'level of evidence' supporting the efficacy of therapeutic interventions; grading the evidence as very high when derived from meta-analyses of well-conducted phase III studies, or from large well-conducted phase III studies relative to lower levels such as that derived from non-randomised studies, anecdote or expert clinical opinion. A major shortcoming of this approach is that it may result in a high level of evidence irrespective of the actual magnitude of the benefit observed, even if the magnitude of the benefit is very limited.[148] This discrepancy has been emphasized by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group which was formed in 2000 to improve the quality of guideline formulation. The GRADE working group emphasised that a particular quality of evidence does not necessarily imply a particular strength of recommendation.[149,150] They have developed and championed a widely endorsed approach emphasising appropriate framing of research and guideline questions,[151] evaluation of the strength of recommendations that incorporates evaluation of the balance between desirable and undesirable outcomes (estimated effects), and the confidence in the magnitude effect of the interventions on important outcomes.[152]

This recommendation can be accomplished by describing both the level of benefit and the level of evidence for recommended therapeutic interventions. For cancer therapies, the ESMO-MCBS scale provides a clear, well-structured and validated mechanism to indicate the magnitude of benefit in addition to the level of evidence that can inform both national and international (e.g. ESMO) guidelines.

Clinical Decision Making. The data encapsulated in ESMO-MCBS scoring can help clinicians to weigh the relative merits of competing relevant therapeutic options in situations in which there is no direct comparative data comparing the available therapeutic options. The grading may also be of benefit in explaining the relative merit of therapeutic options to patients and their families. This information may be especially helpful when treatments incorporate substantial out of pocket costs and the real 'value' of the treatment needs to be candidly addressed to avoid over investment or sacrifice of limited financial resources to pay for treatments that have only limited magnitude of benefit.

Editorial Decisions and Commentaries. The ESMO-MCBS may be of use to editors, peer reviewers and commentators in considering the clinical significance of research findings from randomised clinical studies, cohort studies and meta-analyses with statistically significant positive findings.

Relevance to the ASCO Initiatives

ASCO has undertaken two initiatives to help promote the value in cancer care. The first was a working group to propose new thresholds for the approval of cancer medications.[153] For each of four conditions (metastatic colon cancer, metastatic breast cancer, non-small-cell lung cancer and pancreatic cancer), they have proposed thresholds for meaningful clinical benefit improvement defined in terms of minimal increases in OS (absolute and HR) and also thresholds for minimal increases in surrogate indicators including 1-year survival and PFS. Interestingly, in non-curative therapies, the ASCO recommended thresholds for survival benefit correlate very closely to the thresholds for ESMO-MCBS score of 4–5 (in form 2a) and the recommended thresholds for PFS correlate closely with the thresholds for ESMO-MCBS score of 3–4 which is the highest attainable when the primary outcome is PFS (in form 2b). Secondly, ASCO has developed a Value in Cancer Care Task Force that has been charged with the challenge of developing a framework for evaluating value in oncology. While concurring with ESMO that the evaluation of net clinical benefit is key element in the evaluation of value, ASCO has not yet described their proposed approach to evaluate the magnitude of clinical benefit. A key challenge for the future will be to establish whether there can be harmonisation between the different approaches to value in Europe and the United States.