Another Study Shows ACC/AHA Risk Calculator Overestimates CVD Events

LOUISVILLE, KY — Four out of five cardiovascular risk-prediction algorithms, including the new American College of Cardiology (ACC)/American Heart Association (AHA) risk calculator, overestimate the risk of cardiovascular events, according to the results of a new study^[1].

The latest analysis, which included patients from the Multi-Ethnic Study of Atherosclerosis (MESA), showed that the 2013 ACC/AHA risk calculator, which is designed to assess the 10-year risk of cardiovascular disease and stroke, overestimated the risk of cardiovascular end points by 86% in men and 67% in women. Overall, the ACC/AHA risk score overestimated risk by a net of 78%, report investigators.

"In terms of calibration, the results aren't surprising," senior investigator Dr Michael Blaha (Johns Hopkins University, Baltimore, MD) told heartwire . "Working with MESA and working with other data sets, many people have noted that these risk scores tend to overestimate risk."

The report was published in the February 17, 2015 issue of the Annals of Internal Medicine. Blaha said the calibration of the ACC/AHA risk model—the accuracy of the model when predicting risk—is "moderate" at best. In terms of discrimination, which uses the C statistic to determine how well the model discriminates between high- and low-risk patients, there was little difference between the five risk scores. "We found, overall, the discrimination was moderate again," said Blaha. "It was no better than prior risk scores."

Risk Calculator Challenged from Day One

Based on cholesterol guidelines that were published along with the risk-assessment guidelines, individuals without evidence of cardiovascular disease or diabetes but who have LDL-cholesterol levels between 70 and 189 mg/dL and a 10-year risk of atherosclerotic cardiovascular disease >7.5% are candidates for statin therapy.

Claims that the risk calculator overestimates the risk of cardiovascular events are not new. In fact, just a day after they were launched at the 2013 AHA Scientific Sessions, Drs Paul Ridker and Nancy Cook (Brigham and Women's Hospital, Boston, MA) spoke out, saying that when they tested the ACC/AHA risk-assessment algorithm in three large-scale primary-prevention cohorts—the Women's Health Study (WHS), the Physicians' Health Study (PHS), and the Women's Health Initiative Observational Study (WHI-OS)— cardiovascular risk was overestimated by 75% to 150%.

It also seemed to overestimate risk in another validation cohort from MESA when tested by the guideline authors themselves. At the time, however, only 5-year data was available from MESA, limiting the analysis.

With the 10-year data now available, the researchers tested three Framingham-based scores, including the Framingham Risk Score (FRS) for the prediction of coronary heart disease events (FRS-CHD), the FRS for the prediction of cardiovascular disease events (FRS-CVD), and an FRS modified by the 2001 Adult Treatment Panel cholesterol guidelines (ATP3-FRS-CHD). In addition, the Reynold's risk score, which includes a family history of premature CHD and C-reactive protein (CRP), and the 2013 ACC/AHA risk score were tested. The researchers tested the calibration and discrimination of the five risk algorithms in 4227 MESA participants aged 50 to 74 years old who were followed for 10.2 years.

Overall, the FRS-CHD, FRS-CVD, ATP3-FRS-CHD, and the 2013 AHA/ACC risk scores overestimated the risk of cardiovascular events by 53%, 37%, 154%, and 86%, respectively, in men, and by 48%, 8%, 46%, and 67%, respectively, in women. The Reynold's risk score was the best calibrated model, with investigators reporting the lowest discordance between actual and predicted events (-3%) when tested in MESA. The overestimation of risk could not be explained by differences in the rates of aspirin, statin, or antihypertensive medication use or by differences in coronary revascularization rates, say the investigators.

Predicted and Observed Events with the Risk Scores

Given the findings, Blaha told heartwire that risk scores are very inherently limited because patient populations differ and that there are limitations with simply measuring a cardiovascular risk factor, like blood pressure or cholesterol, at single point in time. The risk calculator, he said, should only be used as a "starting point" for doctors to talk to their patients about treatment.

"I tend to think that the risk score is as useful to patients as it is to physicians," he said. "It's useful for the average person to plug in their numbers and get a rough idea of their risk. When it comes to actually making a decision, you need much better information than just what the risk score provides."

Balancing Overprescription With Drug Risks

As part of the study, the researchers also focused attention on individuals with a 10-year risk of cardiovascular events or stroke >7.5%, as these would be patients who qualify for statin therapy based on the new cholesterol guidelines. For men with a 10-year risk ranging from 7.5% to 10% using the 2013 ACC/AHA risk calculator, the actual event was 3.0%. For women, the actual event rate was 5.1%.

"This is that sensitive population where treatment might be uncertain," said Blaha. "We noticed similar overestimation of risk compared with the rest of the group. That's important because it affects decision making. Now, a patient might say it overestimates the risk a little bit, but I'm high enough risk that I want to take [a statin], or it could change the equation. It needs to be individualized."

In an accompanying perspective written by Ridker and Cook^[2], the editorialists point out that a 7.5% risk of cardiovascular events at 10 years does not automatically mandate the start of statin therapy, but instead, like the guideline authors themselves suggested, is the beginning of a conversation about how best to manage risk.

Ridker and Cook say the "overestimation" issue with the ACC/AHA risk calculator is important, as some patients might end up on lifelong statin therapy. "This is a relevant issue because statins are associated with a small increase in the risk for diabetes," they write. Countering this, statin use might not be such a concern to patients or physicians because studies have shown statins reduce the risk even in patients with low absolute risk of cardiovascular events, "and the benefits of vascular-event reduction seem to exceed the risk for diabetes," they note.

Blaha reports grants from National Institutes of Health/National Heart, Lung, and Blood Institute and personal fees from Pfizer. Ridker reports grants from Amgen, AstraZeneca, Novartis, Pfizer; grants from Donald W Reynolds Foundation; and other fees from Siemens and AstraZeneca. Ridker is listed as a coinventor on patents held by the Brigham and Women's Hospital that relate to the use of inflammatory biomarkers in cardiovascular disease and diabetes that have been licensed to Siemens and AstraZeneca.

Heartwire from Medscape © 2015  Medscape, LLC

Cite this: Another Study Shows ACC/AHA Risk Calculator Overestimates CVD Events - Medscape - Feb 18, 2015.