Many Clinical Trials Unable to Supply High-Quality Evidence

Troy Brown

May 01, 2012

May 1, 2012 — Many interventional clinical trials are small, with significant heterogeneity in the methods used, including blinding, randomization, and the use of data monitoring committees (DMCs), according to a study of clinical trials registered at the National Library of Medicine's The study was published in the May 2 issue of JAMA.

"Our analysis raises questions about the best methods for generating evidence, as well as the capacity of the clinical trials enterprise to supply sufficient amounts of high quality evidence needed to ensure confidence in guideline recommendations," write Robert M. Califf, MD, vice chancellor of clinical and translational research, director of the Duke Translational Medicine Institute (DTMI), and professor of medicine in the Division of Cardiology at the Duke University Medical Center in Durham, North Carolina, and his colleagues.

"Given the deficit in evidence to support key decisions in clinical practice guidelines…as well as concerns about insufficient numbers of volunteers for trials…the desire to provide high-quality evidence for medical decisions must include consideration of a comprehensive redesign of the clinical trial enterprise," they continue. Currently, less than 15% of major guideline recommendations are evidence-based, they write.

On September 27, 2007 (10 years after was established), the researchers downloaded 96,346 clinical trials registered there and entered them into a relational database. They identified interventional clinical trials and selected clinical trials in the 3 areas that make up the largest number of disability-adjusted life-years lost in the United States: cardiovascular, mental health, and oncology.

They first assessed study characteristics overall, then analyzed interventional trials, and finally analyzed 2 temporal subsets: October 2004 through September 2007 and October 2007 through September 2010. Clinical was mandated by Congress in 1997; beginning in 2005, a policy announced by the International Committee of Medical Journal Editors in September 2004 required registration of clinical trials as a prerequisite of publication.

The Food and Drug Administration Amendment Act required trial sponsors or their designees to register trials and document key data elements, effective September 27, 2007. Trial sponsors/designees were required to report basic results effective September 27, 2008, and to report adverse events effective September 27, 2009.

Small, Heterogeneous Trials

Almost all (96%) of the clinical trials had a planned enrollment of 1000 or fewer participants, and almost two thirds (62%) had 100 or fewer participants. The median number of participants was 58 (interquartile range, 27 - 161) for completed trials and 70 (interquartile range, 35 - 190) for trials that have been registered but not completed.

Heterogeneity was evident in the use of DMCs, randomization, and blinding.

After adjusting for 9 characteristics of clinical trials, including phase, number of participants, trial start year, and intervention types, the investigators determined that industry sponsors were far less likely to report the use of DMCs than other types of sponsors. For example, the use of DMCs was more than 9 times more common among National Institutes of Health (NIH)–sponsored clinical trials (adjusted odds ratio [OR], 9.09; 95% confidence interval [CI], 7.28 - 11.34) than among industry-sponsored trials.

Earlier- and later-phase trials were less likely than phase 3 trials to report use of DMCs (adjusted OR, 0.83; 95% CI, 0.76 - 0.91 [earlier phase]; adjusted OR, 0.52; 95% CI, 0.47 - 0.58 [later phase]).

Mental health trials were more likely than cardiovascular and oncology trials to report use of DMCs.

Oncology trials were least likely to use randomization (64.7% didn't use randomization, vs 26.2% for cardiovascular trials and 20.8% for mental health trials), and 87.6% of oncology trials were not blinded.

The researchers found a decrease over time in the numbers of missing data elements for some characteristics. The rate of registered trials failing to report use of DMCs decreased from 57.9% to 18% between the 2 time periods studied; failing to report randomization decreased from 5.6% to 4.2%; and failing to report blinding decreased from 3.5% to 2.7%.

Restoring Public Confidence

In an accompanying editorial, Kay Dickersin, MA, PhD, director for the Center for Clinical Trials at Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland, and Drummond Rennie, MD, professor in the Department of Medicine and Philip R. Lee Institute for Health Policy Studies at the University of California, San Francisco, write, "If all trial investigators and sponsors, as well as the regulators, were equally engaged, this might help further the universal adoption of trial registration; improve the completeness, validity, and accessibility of information in the registries; and restore the public's confidence in investigators, sponsors, and the clinical trial enterprise."

"Trial registration is not some bureaucratic exercise but partial fulfillment of a promise to the patients who agree to participate in these trials on the understanding that the information learned will be made public," Dr. Dickersin and Dr. Rennie write.

Financial support for this project was provided by a grant from the US Food and Drug Administration awarded to Duke University for the Clinical Trials Transformation Initiative. Dr. Califf reports a variety of relevant financial relationships, including with Amylin, Johnson & Johnson (Scios), Merck, Novartis Pharma, Schering Plough, Bristol-Myers Squibb Foundation, Aterovax, Bayer, Roche, and Lilly. Dr. Kramer also reports a variety of relevant financial relationships. Dr. Dickersin and Dr. Rennie have disclosed no relevant financial relationships. Full disclosures are listed on the journal's Web site.

JAMA. 2012;307:1838-1847.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.