Can Results of Randomized Trials Be Generalized to Real Life?

January 12, 2007

January 12, 2007 (Paris, France) - MI patients participating in randomized controlled trials have a lower baseline risk and experience lower mortality than nonenrolled eligible patients, a new study shows [ 1].

The study, published in the January 8, 2007 issue of the Archives of Internal Medicine, found that the lower mortality in trial patients was not entirely explained by differences in baseline risk, use and type of reperfusion therapy, and/or delays in presentation. Caution is therefore necessary when the findings obtained in controlled trials are extended to the general population with MI, the authors say.

Lead author Dr Philippe Gabriel Steg (Bichat Hospital, Paris, France) commented to heartwire : "We found that patients from real life who would be eligible for clinical trials do not actually behave like participants in randomized controlled trials. While this does not imply that the directional effects of therapies tested in randomized trials would differ, it does highlight an issue with the generalizability of randomized trials. This is why it is important to design trials that are widely generalizable, with broad population inclusion criteria, and to complement randomized trials with observational studies."

Highly selected populations

In the paper, Steg et al explain that randomized controlled trials provide the foundation for evidence-based medicine, but the relevance of these trials to clinical practice may be hampered because they tend to recruit highly selected populations that may not be representative of patients encountered in everyday practice. It therefore remains unclear whether patients enrolled in such trials accurately represent individuals with similar conditions routinely encountered in practice. Noting that few data are available to compare trial participants with trial-eligible or trial-ineligible patients in the same setting over a similar period, they set out to do just that using data from the GRACE registry of patients hospitalized with a suspected acute coronary syndrome.

This analysis included 8469 patients in the GRACE registry with a confirmed diagnosis of ST-segment-elevation MI, who were divided into one of the three following groups: current participants in a randomized controlled trial; not enrolled in a randomized trial but meeting the eligibility criteria for the ASSENT- 3, GUSTO-V, or DANAMI- 2 trials; or not enrolled in a trial and ineligible for participation in any of these three trials. These three groups of patients were treated in the same hospitals, by the same teams, and over the same period.

Results showed that rates of mortality, cardiac arrest, and cardiogenic shock were all much lower in the patients participating in a randomized trial compared with those who were eligible but did not participate, and (as expected) the highest rates occurred in the ineligible patients.

Rates of mortality, cardiac arrest, and cardiogenic shock according to whether included or eligible for randomized trials

Event Patients in a randomized trial (n=953) Patients eligible but not in a randomized trial (n=4669) Ineligible patients (n=2847)
In-hospital mortality (%) 3.6 7.1 11.4
Postdischarge mortality (%) 3.0 4.8 7.7
Cardiac arrest (%) 5.1 7.4 9.7
Cardiogenic shock (%) 4.1 6.1 8.8

 

The same gradient of mortality was seen in each risk tertile and across age and sex categories, with the lowest mortality always occurring in patients participating in randomized trials and the highest mortality always occurring in ineligible patients.

Multivariable analysis adjusting for baseline risk, use and type of reperfusion therapy, and delay from symptom onset to admission consistently showed a higher mortality rate for eligible nonenrolled patients than for trial participants (odds ratio 1.97, 95% CI 1.24-3.13).

The researchers note that the proportion of eligible patients who did not take part in a trial was relatively high, reflecting the fact that the three randomized trials considered were pragmatic in their enrollment strategy. But even so, there remain important differences in baseline characteristics, baseline risk, and outcomes between eligible patients and actual participants, with hospital mortality doubled in unenrolled eligible patients compared with trial participants. "It is unknown whether the benefits of therapies demonstrated in trial participants can be extended safely to populations so markedly different in terms of their risk and outcomes," they write.

The authors suggest several possible reasons for the different risk-adjusted outcomes seen in this study, including a possible beneficial effect of the intervention tested in the trial and overall better care in the trial patients, supported by a gradient in the use of evidence-based therapies and procedures from trial participants to eligible and ineligible patients. But they add that residual confounding variables may also account for the differences.

Steg et al believe this analysis should not be viewed as detracting from the value of randomized controlled trials but rather as providing an example of the lingering problem with the generalizability of such trials. "Because eligible patients differ markedly from trial participants and because one third of patients are ineligible, with completely different baseline characteristics and outcomes, the outcomes of randomized controlled trials should always be extrapolated with caution to real-life patients," they say, adding that continued efforts are needed to improve the external validity of randomized trials, including simplification of enrollment criteria and inclusion of patients seen in routine practice.

  1. Steg PG, Lopez-Sendon J, Lopez de Sa E, et al. External validity of clinical trials in acute myocardial infarction. Arch Intern Med 2007; 167:68-73 .

The complete contents of Heart wire , a professional news service of WebMD, can be found at www.theheart.org, a Web site for cardiovascular healthcare professionals.

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as:

processing....