Effectiveness in the Absence of Efficacy

Cautionary Tales From Real-World Evidence

Safiya Karim, MD; Christopher M. Booth, MD


J Clin Oncol. 2019;37(13):1047-1050. 

In This Article


Real-world data (RWD) are collected outside of traditional clinical trials and may include electronic health records, patient registries, and administrative health care claims.[1,2] Real-world evidence (RWE) is derived from analysis and aggregation of these data. Published reports of RWE have increased substantially in recent years. A PubMed search on the terms "real world data," "real world evidence," and "registry" showed a 600% increase in citations during the period 2002 to 2016 (from 2,435 citations per year to 14,956 citations per year). RWD have historically been used to answer questions related to trends in cancer incidence and mortality; quality of and access to care delivered in routine practice; outcomes of rare cancers; and understanding the incidence of rare events and toxicities in the general population. There is growing interest in the use of RWD to study effectiveness of treatments in the real world (ie, comparative effectiveness research [CER]). Although other reports have highlighted the benefits and some pitfalls of RWE,[3–7] in this commentary, we highlight a specific scenario in which readers of CER should be cautious in their interpretation of reported results. We wrote this commentary because such studies are increasingly common and have the potential for patient harm if therapies are adopted solely on the basis of analyses of RWD. This is particularly important because there are signals that regulatory agencies may begin approving drugs on the basis of observational data.[8]