Sizing Up ESMO's Drug Benefit Scale

David J. Kerr, CBE, MD, DSc, FRCP, FMedSci; Nathan I. Cherny, MD


October 12, 2015

This feature requires the newest version of Flash. You can download it here.

David J. Kerr, MD: Hello. I'm David Kerr, professor of cancer medicine at the University of Oxford. Welcome to this edition of Medscape Oncology Insights.

Today we're going to talk about assessing the relative value—and I use that word quite guardedly and specifically—of treatments for individual patients. The European Society for Medical Oncology (ESMO) has created the Magnitude of Clinical Benefit Scale, or MCBS, to help us do that.

I'm absolutely delighted to be able to welcome my friend and colleague of many, many years—Professor Nathan Cherny, Norman Levan Chair for Humanistic Medicine at Shaare Zedek Medical Center in Jerusalem, and associate professor of medicine at Ben-Gurion University of the Negev in Beersheba, Israel. Welcome, Nathan.

Nathan I. Cherny, MD: Thanks.

Dr Kerr: Tell us about this tool. You and I have been talking about this for over a decade now. How we can start to rank the benefit of individual drugs for patients? This might have implications for treatment guidelines, how governments respond, and so on. So it's politically—potentially—a bombshell of a tool.

Dr Cherny: Well, it's all of those and something more as well. The something more really comes from the problem of hype. Often, with drugs that are developed or licensed, there's a lot of hype about their effectiveness—sometimes overstatements, sometimes exaggeration—which is harmful. When effectiveness is exaggerated, it is harmful because it tempts either patients and their families or healthcare systems to allocate resources to expensive medications that may not deliver. So, we really needed a very honest way to sort out the drugs that can deliver in a big way from those drugs that provide a minimal amount of benefit.

How Much Benefit Can the Drug Deliver?

Dr Kerr: Is this the bare bones of a cost-effectiveness analysis? Is this the burden of toxicity?

Dr Cherny: It's both of these. Value is a relationship between the magnitude of benefit and the cost. Costs vary from country to country, and out-of-pocket costs vary between healthcare systems. So, we set cost aside.

What is universal, and what's going to be the same in every setting, is how much benefit the drug can actually deliver. This is what we needed to sort out for individual patients, for policy-making decisions, and for health technology assessments. Given the complexity and the varying outcomes that clinical studies currently generate, we needed a single platform—a way to scale the relative benefit of drugs, even though we're often comparing apples with oranges in terms of different outcome measures.

Dr Kerr: You know that I work within a socialized healthcare system in the United Kingdom. We've got NICE—the National Institute for Health and Care Excellence (some would say cost-effectiveness)—and we have a transparent, formulaic approach to dealing with new drugs. How much benefit do they bring, and does it come at a cost that we as tax-payers can afford? Is it a similar methodology? Does MCBS work something like that?

Dr Cherny: What we've done here is we've developed a uniform and reproducible methodology to be able to give a relative scaling of the clinical benefit side. Now, once you've shown clinical benefit, the issue of value is still an open question, depending on what the drug costs. Each healthcare system will make its own decision as to how much they're prepared to pay for X amount of additional time or X amount of increased overall survival.

Dr Kerr: If health policy-makers have access to the tool, will health policy-makers in New York, Paris, or Dar es Salaam be able to use this? Is it sufficiently scalable?

Dr Cherny: Absolutely— yes, it is. In terms of the data on clinical benefit, it is usable anywhere. It also needs to be tied to local factors, such as cost and disease prevalence. Then, with those two sides of the issue, you're able to make an assessment.

Dr Kerr: Are there winners and losers? Is there a black list? Is there a gold list?

Dr Cherny: So far, in developing the tool, we field-tested it against 77 different drugs in 10 different conditions. Then we correlated that with expert reviews. In fact, we have over 300 peer reviews. So, it's really been very highly validated.

The tool has been able to sort out the outstanding drugs, the strong drugs, and the drugs that show fairly marginal levels of benefit. That will hopefully help the process of health technology assessments on one hand.

And when I'm sitting across from a patient who says, "What else is there that I can try?" I'll be able to give at least a numerical value as to whether a drug can give the patient significant benefit or just a very small amount of benefit. Some people will say, if the benefit is that small and the drug is not covered by my insurance or by the National Health Service (NHS), this is not worth it for me or my family. That's important information.

Which Drugs Get a Thumbs-Up?

Dr Kerr: I agree entirely. We know that in the United States, the American Society of Clinical Oncology (ASCO) is getting behind the concept of value. So there are more and more of us clinical academics prepared to stick their heads above the parapet in this thorny issue.

Let me be provocative if I may and ask you to name names. What drugs have been designated as being of marginal benefit?

Dr Cherny: I'll bring in three examples. One of the most striking ones—and everyone has been aware of this for a long time—is the role of erlotinib (Tarceva®) in pancreatic cancer, where it showed overall a 9-day median survival advantage.[1] Although it was statistically significant and it can score level 1 evidence for benefit, the quantity of benefit was really very, very small, and I would go as far as to say clinically insignificant.

Dr Kerr: Agreed.

Dr Cherny: One of the controversial drugs in oncology has been the role of bevacizumab (Avastin®) in metastatic breast cancer. Here we had a drug that showed a significant progression-free survival advantage in a non-crossover study,[2] but no survival advantage, and when they looked at quality of life there was no improvement.[3]

Using the MCBS, bevacizumab scores the top mark for progression-free survival, which is 3 out of 5, but it is penalized because it showed neither a survival advantage nor improvement in quality of life. Bevacizumab ends up with a score of 2, which on this 5-point scale is not a drug that is likely to be promoted as a priority for a healthcare system.

Another drug that had the same sort of scoring pattern was everolimus in breast cancer—where again, there was a significant progression-free survival advantage, but no overall survival advantage and no improvement in quality of life.[4] Progression-free survival is a surrogate endpoint, and this is why the score is a maximum of 3; it's a weak surrogate for survival and quality of life. If you can't verify either of them...

Dr Kerr: There you go.

Dr Cherny: In contrast, we have a lot of the tyrosine-kinase inhibitors (TKIs) in lung cancer. These were the subject of many crossover studies, so you didn't see a survival advantage.[5,6,7] The studies were powered for progression-free survival, and the TKIs were scored for maximal progression-free survival. But the investigators evaluated quality of life as well, and the studies showed repeatedly either improved quality of life or delayed deterioration in quality of life. So TKIs get bumped up from 3 to 4, which puts them in the bracket of priority drugs.

MCBS May Raise the Bar for Clinical Trial Design

Dr Kerr: What do we do with this information? Do we take it to health administrations around the country? Do we take it to the World Health Organization (WHO)? Do we incorporate it into ESMO guidelines? What comes next?

Dr Cherny: ESMO is formulating its policies about how they're going to use the tool, but I think the plan at this stage is to incorporate the tool into the ESMO guidelines. We've put this out in the public arena and we're presenting it in multiple forums, so we really anticipate that many different people are going to see utility in it—including the pharmaceutical manufacturers, because if they can see that the design in their drug trials is not going to be able to get them a potentially high enough score, then this may influence the design of trials.

As things are now, many new agents are coming in on the lowest branch—that is, on the progression-free survival advantage alone. For such organizations as the US Food and Drug Administration (FDA) and the Medicines and Healthcare products Regulatory Agency (MHRA), this is picking low-hanging fruit. We've said that a progression-free survival advantage is not enough; unless you can show with that you're improving quality of life (by incorporating quality of life as an endpoint and showing that quality of life either gets better or that deterioration is delayed), you're not going to make the cut.

Dr Kerr: Do you make a comment on cost?

Dr Cherny: No. We're not addressing the cost issue, because the cost varies from place to place. How much a society or a country can afford to pay for a certain level of benefit is going to be very different between high-income countries, upper-middle-income countries, and low-income countries. Where the range of prices between countries is controlled in a relatively narrow band, as it is for the most part, each country is going to have to make its own decisions about what it can afford and what it can't.

Dr Kerr: Okay, so that may be something. You set the international guidelines, but nationally, countries can take the tool and determine their bandwidth is X, Y, or Z.

Dr Cherny: Yes—this is what we can afford, but in terms of setting the priority of drugs, we're going to try to get [them] into our basket of services.

Dr Kerr: The tool focuses attention on those drugs that we think would deliver most benefit for patients and overall value.

Dr Cherny: Yes. And there's a secondary gain: The integrity and the credibility of the oncology profession are harmed by hype.

Dr Kerr: Agreed—but it's not often generated by us.

Dr Cherny: Well, if you remember the hype around some of the innovations at the last ASCO meeting,[8] which were touted and made headlines as major breakthroughs—when you looked at the data, with very expensive combinations of treatment, all they had shown was progression-free survival.

Dr Kerr: I see the point. So, we're sort of complicit—a strong word—but complicit in that promulgation.

Dr Cherny: Absolutely. By taking a firm stand to say that we're going to bring some discipline to this process, we're enhancing our professional integrity and enhancing the status of the profession.

Dr Kerr: We would say that it's sort of coming together—personalized and public medicine, we're bringing these two together. It's been fantastically interesting speaking to you, as always. It's been absolutely brilliant.

To all of those who've been watching, thank you very much indeed. This is Nathan and I, reporting from the European Cancer Congress in Vienna, 2015. Thanks very much indeed.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: