COMMENTARY

Evidence-Based Medicine and the Cochrane Collaboration on Trial

David K. Cundiff, MD

Disclosures

June 12, 2007

The Cochrane Collaboration was founded in 1993 and was named for the British epidemiologist Archie Cochrane, MD (1908-1988). At the initiation of the National Health Service in the 1940s in the United Kingdom, Dr. Cochrane suggested that the free care in the National Health Service should be limited to interventions supported by evidence of their effectiveness. This was a very innovative thought in the 1940s. Dr. Ian Chalmers, the driving force behind the Cochrane Collaboration, was strongly influenced by the writings of Archie Cochrane.

Most physicians perceive that "evidence-based medicine" means practicing medicine by using tests and treatments that have been vetted by randomized controlled clinical trials (RCTs) to prove that they work. Although the Cochrane Collaboration focuses mainly on systematic reviews of RCTs, it also addresses issues specific to reviewing other types of evidence when relevant.[1]

Bernadine Healy, MD, former Director of the National Institutes of Health, wrote an essay entitled "Who Says What's Best?"; the piece, critical of evidence-based medicine, appeared on September 11, 2006 in US News and World Report.[2] Kay Dickersin, PhD, the Director of Cochrane's US Center for Clinical Trials, asked me and other Cochrane Collaboration evidence-based medicine reviewers to respond by sending op-ed pieces to the media.

I agree with many of the points made by Dr. Healy. She writes that the rigid adherence to RCTs as the only valid form of scientific evidence about medical tests and treatments unfairly excludes other ways of determining what medical interventions will best serve an individual patient with a particular health problem. Dr. Healy put it succinctly: "The autonomy and authority of the doctor, and the subsequent variability in care, are the problems that evidence-based medicine wants to cure."

While I concur with Dr. Healy that relying solely on evidence-based medicine to determine medical guidelines better suits the needs of governments and insurance companies for cost control than the needs of patients for optimal medical treatment, I agree for a different reason. While evidence-based medicine is absolutely essential to comprehensive healthcare reform, it has been profoundly corrupted by money.

In 2001, I volunteered to help conduct a review for the Cochrane Collaboration of the evidence base for the use of anticoagulants (warfarin [Coumadin] and heparins) for blood clots in the lungs (pulmonary emboli, or PE) and legs (deep venous thromboses, or DVT), as recently affected Vice President Dick Cheney. Dr. Juliet Manyemba, a physician from England, and John Pezzullo, PhD, a retired biostatistician formally from Georgetown University School of Medicine, were my coauthors. I disclosed to the Cochrane editor that my research interest in anticoagulants for DVT and PE originated because of a malpractice case against me concerning a DVT patient that resulted in the loss of my medical license.

Warfarin, heparin, and other anticoagulant drugs have been used to treat blood clots since the 1940s based on unscientific anecdotal evidence and observational studies with historical controls. Subsequently, countless RCTs involving blood thinners for venous thromboembolism (VTE) patients have included no un-anticoagulated control subjects. According to anticoagulation researchers contracted or employed by drug companies, the rationale is that it would be "unethical" not to give anticoagulants to clinical research subjects with blood clots in their legs or lungs. After reading over 1000 studies on the topic, I found 1 randomized and properly controlled trial of DVT patients. Patients in the study received either standard treatment (warfarin and heparin) or phenylbutazone (an anti-inflammatory drug like aspirin). From this single well-designed study of anticoagulants came a startling result: The anticoagulants did not prevent deaths.[3,4]

Cochrane archivists turned up 2 other RCTs of anticoagulation therapy in DVT patients. Neither trial found any benefit due to anticoagulants. Summing the results of the 3 trials, 66 DVT patients received anticoagulants and 6 of them died; 60 DVT patients did not receive anticoagulants, and 1 of them died. None of the 3 trials had been referenced in any journal articles or reviews of anticoagulant therapy that I read.

Although these 3 trials show a trend suggesting that warfarin and heparin do harm, there were too few patients to show with statistical significance that anticoagulants increase deaths. But they contain enough subjects to show that anticoagulants do not reduce mortality. Based on the complication rate of anticoagulation for DVT or PE in much larger observational studies, anticoagulants kill 1000-4000 Americans with VTE each year due to internal bleeding, mostly in the brain.[5,6,7] According to a recent population-based study of anticoagulation-related intracerebral bleeding (AAICB) rates in the greater Cincinnati area, AAICB occurred in 2000-2500 VTE American patients in 2004 (5.1-6.5 AAICB cases per 100,000 population; 12.9% of AAICB cases were related to anticoagulation of VTE patients). About 60% of AAICB patients die within 1 year.[8] Most of the surviving 40% remain permanently disabled. Warfarin distribution in the United States quadrupled on a per-capita basis between 1988 and 1999. In Cincinnati, cases of AAICB rose 5.5-fold from 1988 to 1999.[9]

The Cochrane peer reviewers (at least 4 out of 7 of which had undisclosed financial ties to the drug companies that make anticoagulants) delayed four years over releasing this review for publication. When the only 3 RCTs discovered showed no benefit and possible harm from anticoagulants, the editor and peer reviewers directed us to include 8 additional lines of evidence supporting anticoagulation from about 50 other studies in the medical literature. When my critique of those 8 lines of evidence showed that they were all faulty, the peer reviewers did not rebut a single point. Instead, the editor demanded that we delete the additional lines of evidence from the review, because they were not from RCTs. The Cochrane editor also would not allow publication of the estimate of major and fatal bleeding from anticoagulants for VTE, because I derived those figures from large retrospective observational studies and not RCTs.

When the author of one of the randomized trials discovered by the Cochrane archivists refused to cooperate and clarify to us his method of randomizing patients in his study, the Cochrane editor and/or peer reviewers invented a reason to disqualify the trial from inclusion in our review. The editor told us to accept the edits or the review wouldn't be published. The "authors' conclusions," written into our article by the Cochrane editor and peer reviewers, were these: "The limited evidence from randomized controlled trials of anticoagulants versus nonsteroidal anti-inflammatory drugs or placebo is inconclusive regarding the efficacy and safety of anticoagulants in venous thromboembolism (DVT and PE) treatment. The use of anticoagulants is widely accepted in clinical practice, so a further randomized trial comparing anticoagulants to placebo could not ethically be carried out."

In our final draft of the review, we authors said that a placebo-controlled trial would be impractical and suggested a "noninferiority trial" with anticoagulants vs a nonsteroidal anti-inflammatory drug. One of the peer reviewers, with no conflict of interest that I could find, commented, "Note that it is ethically possible to conduct a study to determine if anticoagulation therapy is harmful. If nothing else, dose reduction studies could determine if lower doses or weaker therapies (aspirin or NSAIDS?) are equally effective" (ie, a noninferiority trial).

Finally, in January 2006, The Cochrane Database of Systematic Reviews published our review -- completely altered by the peer reviewers and editor -- entitled, "Anticoagulants or Non-steroidal Anti-inflammatories or Placebo for Treatment of Venous Thromboembolism."[10] At the suggestion of Dr. Dickersin, I issued a complaint to the Cochrane Collaboration publication arbitrator in September 2006. In the 7 months since I submitted the complaint, 580-2300 American VTE patients have bled to death from anticoagulants as estimated from observational studies.[5,6,7] This estimate is consistent with 1160-1450 deaths based on the above mentioned Cincinnati population-based AAICB study.[8] Worldwide, at least twice as many have died.

Dr. Dickersin recently told me that she cannot estimate when the Cochrane investigation will be completed. A recent similar investigation took 2 years. The publication arbitrator has resigned, and she has no guarantee of finding a replacement anytime soon. Options that they are considering include withdrawing the review from the Cochrane Database of Systematic Reviews and having new peer reviewers critique the review (except that she doesn't know when she can find new peer reviewers with expertise in anticoagulation). Options that are not under consideration include printing the review as we authors wrote it, having the Cochrane peer reviewers reply to my analysis of their 8 lines of evidence supposedly supporting anticoagulation, and asking the FDA Office of Surveillance and Epidemiology (formerly Office of Drug Safety) to examine the evidence and issue a report, as I suggested 7 months ago.

Medscape General Medicine published my entire review, including the evidence from the 8 lines other than the 3 RCTs included in our Cochrane VTE review.[11,12,13] The Medscape VTE review concluded, "Anticoagulants have not been shown to be efficacious in reducing morbidity or mortality or safe in venous thromboembolism treatment." Medscape Chief Editor, George Lundberg, MD (former Chief Editor of JAMA), wrote the accompanying editorial, entitled "Is the Current Standard of Medical Practice for Treating Venous Thromboembolism Simply Wrong?"[14] I sent the link to the review to several anticoagulation experts at the Food and Drug Administration. Warren Rumble, Ombudsman - FDA, Center for Drug Evaluation and Research, replied for them: "Thank you for asking FDA to provide an official response to your published article in Medscape General Medicine. I have consulted with officials in our Center for Drug Evaluation and Research regarding your request, and we will not have a response to your publication. We appreciate that you provided access to your article, and the chance to comment on it."

Since anticoagulation researchers and FDA scientists chose not to rebut any of the data or conclusions of either review, the media was not interested, few physicians read the reviews, and no debate ensued. Researchers continue receiving lucrative contracts from drug companies for more anticoagulant trials. The medical establishment (drug companies, doctors, hospitals) keeps making money from the diagnosis and treatment of DVT and PE with anticoagulants (estimated total cost in 2007 will be $13 billion-$48 billion in the United States[15]), and medical journals keep publishing more anticoagulation trials without proper controls, which are dutifully covered by a compliant media, while thousands of DVT and PE patients keep bleeding to death.

To have a fair resolution concerning the interpretation of the scientific evidence about the efficacy and safety of anticoagulants for the treatment of VTE, I suggest that the reader call for the FDA Office of Surveillance and Epidemiology to investigate the issue and decide which of the 2 reviews of this topic published under my name[11,10] is valid (Office of Surveillance and Epidemiology Director Gerald Dal Pan, MD, gerald.dalpan@fda.hhs.gov ; Ph: 301-796-2380; or go to http://www.fda.gov/cder/comment.htm).

 


Readers are encouraged to respond to the author at dkcundiff3@verizon.net or to Paul Blumenthal, MD, Deputy Editor of MedGenMed, for the editor's eyes only or for possible publication via email: pblumen@stanford.edu

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as:

processing....