A few months ago, I gave a talk as part of a panel I organized on speeding access to promising therapies in oncology.
I discussed the concept of "in silico trials" — using real-world data to emulate target trials. The goal was not to replace phase 3 trials but rather to inform the power and efficiency of those trials. I aimed to showcase the potential of applying several strategies, like using machine learning on representative observational data, to help design better trials.
The speaker after me, whom I'd invited, was meant to discuss challenges with using observational data to emulate clinical trials.
The first line of their talk: "I disagree with basically 100% of everything the previous speaker said."
The speaker then proceeded to take down the concept of emulated trials by arguing that observational data frequently — and perhaps dangerously — disagrees with results from clinical trials. The talk was full of case examples that called out many seminal observational studies as well as their authors for engaging in observational research.
Now, to be fair, the lecture was meant to be a counterpoint to mine, and I knew the speaker believed strongly that observational data were flawed. I also didn't mind the passionate tone of the lecture; it provided some entertainment for what otherwise could be a boring topic. And perhaps most important, the lecturer provided many good points that I have used to qualify my own future arguments for in silico trials.
But the talk also encapsulated much of what has bothered me about the evidence-based oncology movement over the past 5 years. The evidence-based oncology movement, which started with good descriptions of principles that clinical trials ought to follow, has now transformed into short-form takedowns of trials, investigators, and the FDA. This approach alienates individuals — trialists, pharma, government regulators — with whom evidence-based medicine enthusiasts have to work with closely to get anything done.
This takedown approach has become particularly evident through hyperbolic, often disrespectful statements, often said to gain likes instead of provide nuance. So much of this megaphone is devoted to arguing why drugs should be taken off the market, why trials should never have been done, or simply to insult the FDA or clinical trialist. Many of these statements come from well-respected clinicians.
I came across a recent example posted on social media: "The modern role of an oncology clinical trialist is to fraternize with pharma, run their trials, fill out the forms they give you, enroll patients, and sign off on their manuscript (written by a medical writer)."
Now, granted, I'm not a traditional clinical trialist. But I interact with them regularly. The statement above is just wrong. Trialists write investigator-initiated trials or serve as principal investigators for phase 3 trials. They spend hours — often in evenings outside of patient care — writing protocols, managing research staff, collecting specimens, and coordinating their site investigators. On top of that, they help identify and manage adverse events as well as determine the next best treatments for patients.
Painting these oncologists as shills for pharma is more than just incorrect; it serves nobody. It makes patients distrustful of their oncologists and diminishes trust at a time when doctors have been generally dragged through the mud.
It also leaves me wondering, will evidence-based oncology enthusiasts ever see a trial or drug that they like?
For many who run clinical trials of investigational agents or who are responsible for drug approvals, the answer is probably no. There is little nuance to these takedowns. The tone is too vitriolic. And, frankly, many of these individuals would probably prefer to tweet or release a podcast rather than pick up the phone and call investigators or policymakers to describe their views.
In short, the modern evidence-based oncology movement has succeeded in bringing awareness to the perils of observational data, surrogate endpoints, and faulty control arms, which is a good thing. But the delivery of this message has come at a price. It has created division among researchers, policymakers, and patients and ultimately may be harmful to patients.
So, what is to be changed about the evidence-based oncology movement?
When I was a master's student, I took an interesting class in persuasion by Gary Orren. The class was distinct from academic writing, policy analysis, or negotiation. The point was to teach lessons in how to persuade a spectrum of people to make change.
We learned about key concepts of persuasion: logos (an appeal to logic), pathos (an appeal to emotion), and ethos (an appeal to credibility and authority).
Many evidence-based oncology enthusiasts rely on logos arguments. Their academic manuscripts and tweets are rife with examples of flawed trial design or scientific conclusions from hundreds of studies. They frequently justify these claims with rigorous meta-analyses or systematic reviews highlighting how oncology trials have prioritized publishing a positive result over generalizable evidence. I have participated in similar studies or viewpoints.
But these appeals to logos have come at the expense of arguably more important principles for changing minds and bringing about change: ethos and pathos.
First, ethos: Many evidence-based enthusiasts are dry lab scientists or clinicians and do not specialize in enrolling patients in clinical trials or participating in FDA advisory panels. As such, their arguments may be less compelling because a common perception is that these individuals do not know "how the sausage is made" and their arguments are too idealistic. For instance, in the takedown of in silico trials, the lecturer said it would be fantastic to have a world with a randomized trial for every single clinical question in oncology.
Unfortunately, this idealism ignores the fact that clinical trials are expensive and often depend on fickle federal funding or industry agents who are not interested in designing the most generalizable trial. And this is where observational data can fill in knowledge gaps that phase 3 trials do not answer.
As for pathos, this is where I think evidence-based enthusiasts fall the shortest. When building a movement, there is a real opportunity to engage patients, advocacy groups, and other trialists.
What about the patient who received a drug that should have never been approved? Or the oncologist who witnessed their patient incur a terrible side effect from a novel agent that showed, at best, marginal benefit in clinical trials? What of the patient who decided to participate in a clinical trial who died after receiving an inferior therapy because the crossover to a standard-of-care drug was not allowed?
These examples should be key to evidence-based enthusiasts' arguments but rarely make it in their podcasts, tweets, or lectures. I'm not sure why that is but I do know that there is a limited extent to which logic-based arguments can move the needle on changing minds or practice.
Above all else, I see two main things that evidence-based oncology enthusiasts ought to consider to make change.
First, these enthusiasts should target their message to individual stakeholders rather than shout at the wind. Simply arguing that a trial design is flawed or that surrogate endpoints are bad may not prompt changes because they do not persuade the stakeholders who need to hear the message.
For example, pharma executives, who are incentivized by market share and first-to-market drugs, are motivated to accelerate their own products to market by using surrogate endpoints. Generally pointing out that this practice is flawed probably will not influence how these executives behave. Instead, making a financial argument that designing better trials could improve their bottom line in the long run may be more convincing.
Medical residents and fellows, who are incentivized by learning statistical methods and generating publications, may be motivated to conduct observational data analyses, even when these analyses may not be replicated by a clinical trial. Perhaps the argument to these individuals shouldn't be that all observational studies are flawed, but rather that taking the time to learn clinical trial methodology or to participate in clinical trial recruitment carries large dividends for research and patients in the long run, even if a publication does not result.
Second, evidence-based enthusiasts need to acknowledge uncertainty and gray zones. There is often an "I told you so" tendency in tweets when a certain drug fails or is withdrawn from the market. I think a better approach is to acknowledge that drug effectiveness is rarely all-or-none. For some percentage of patients, investigational agents work. For others, they don't. This uncertainty is often what we, as oncologists, are trying to manage when we decide to use certain agents like immunotherapy, which have response rates under 50% in many cancers but can have durable responses.
Rather than push for wholesale withdrawals of drugs, as an oncology community, we should champion low-cost diagnostics that can risk-stratify populations into likely vs not likely responders.
Overall, the evidence-based oncology movement could benefit from a bit more humility. Most trialists, policymakers, and patients will be less likely to engage with or be convinced by a perceived know-it-all.
I believe in evidence-based oncology. I think trials should evolve and adopt common standards so we know that we can trust the results. I think observational data has its flaws and is nearly always trumped by a well-designed, generalizable clinical trial.
But I also see that the evidence-based oncology movement has been co-opted by a small number of voices who seem more interested in creating negativity and skepticism around clinical trials instead of providing nuanced arguments and actionable solutions. And this strategy, in addition to being harmful to patients, probably won't persuade change.
Lead image: Hoag Levins
Medscape Oncology © 2023 WebMD, LLC
Any views expressed above are the author's own and do not necessarily reflect the views of WebMD or Medscape.
Cite this: Why Evidence-Based Medicine Enthusiasts Could Use a Lesson in Persuasion - Medscape - Feb 16, 2023.