Evidence Underlying Most Guidelines May Be Questionable

Laurie Barclay, MD

January 21, 2014

Most clinical practice guidelines for interventional procedures are based on lower-quality medical evidence and fail to disclose the authors' conflicts of interest, according to a review and analysis published in the January issue of the Mayo Clinic Proceedings.

"Guidelines are used as a means to establish a standard of care," write Joseph D. Feuerstein, MD, from the Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, and colleagues. "This standard of care assumes that the guidelines are fundamentally sound and supply the framework for providing exceptional care. However, a guideline's validity is rooted in its development process."

Limitations of guidelines development include the absence of evidence to support recommendations; reliance solely on expert opinion, particularly when those experts have pertinent disclosures that could create bias; and insufficiently frequent updates because of cost and time constraints.

The researchers examined the validity of guidelines published as of November 15, 2012, on the Web sites of interventional medical societies: the American Association for Bronchology and Interventional Pulmonology, American Society of Diagnostic and Interventional Nephrology, American Society for Gastrointestinal Endoscopy, and Society for Cardiovascular Angiography and Interventions.

Because the American Association for Bronchology and Interventional Pulmonology did not publish guidelines, the reviewers also assessed guidelines from the American Thoracic Society and American College of Chest Physicians. Criteria reviewed were graded levels of evidence, methods used to grade the evidence, and disclosures of conflicts of interest (COIs).

Grading of Evidence Lacking

The researchers evaluated 153 interventional guidelines, of which 4 were duplicates. The mean age of the guidelines was 5.2 years.

Less than half (46%; 69/149) of the guidelines graded the quality of evidence, using 7 different methods. Grading was published in 46 (71%) of 65 American Society for Gastrointestinal Endoscopy guidelines, in 23 (29%) of 78 Society for Cardiovascular Angiography and Interventions guidelines, and in none of the 4 American Society of Diagnostic and Interventional Nephrology or the 2 pulmonary society guidelines.

Of the 3425 recommendations reviewed, level A evidence supported only 11% (n = 364), whereas level B evidence supported 42% (n = 1432) and level C evidence supported 48% (n = 1629).

Nearly two thirds of the guidelines (62%; 92/149) failed to comment on COIs. When information regarding disclosures was published, 91% of guidelines (52/57) reported COIs. A total of 1827 COIs were reported by 45% of the authors (317/697), yielding an average of 5.8 COIs per author.

"Most of the interventional guidelines failed to grade the evidence," the review authors write. "When present, most guidelines used lower-quality evidence[, and] most guidelines failed to disclose COIs. When commented on, numerous COIs were present."

Limitations of this review include the use of multiple grading systems by the various societies and review of only the guidelines published on the society Web sites.

Need for Greater Uniformity in Guidelines Development

"Future guidelines should clearly state the quality of evidence, use a standard grading system, be transparent regarding potential biases, and provide frequent updates," the review authors conclude.

"A more uniform and standardized process is necessary for the development and maintenance of practice guidelines across all societies. Given the potential issues raised in this study, clinicians should be prudent regarding the use of current societal guideline recommendations."

In an accompanying editorial, Jayant Talwalkar, MD, associate medical director of the Value Analysis Program in the Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Rochester, Minnesota, notes that the lack of attention to individual and organizational COI may be the greatest threat to creating trustworthy clinical practice guidelines.

"Clinical practice guidelines are clearly a part of the fabric defined by evidence-based medicine and the pursuit to achieve safe, efficient, and patient-centered delivery of medical care," Dr. Talwalkar writes. "[This study] further highlights that existing guidelines are highly variable with respect to high quality and transparency, but these problems can be overcome with further education and adherence to emerging standards in the field."

One coauthor has served as a consultant for or has received grants from Prometheus Laboratories, Alba Therapeutics, Alvine Pharmaceuticals, and Shire. One coauthor has served as a consultant for or has received grants from Abbott Laboratories, Janssen Pharmaceuticals, Warner Chilcott, Given Imaging, Prometheus Laboratories, and Pfizer. The other author and Dr. Talwalkar have disclosed no relevant financial relationships.

Mayo Clinic Proc. 2014;89:16-24. Article full text, Editorial full text


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.