DALLAS, TX – Is the new calculator for assessing the 10-year risk of atherosclerotic cardiovascular disease flawed, as suggested by a recent news article, or does it work exactly as the committee members of the risk-assessment guidelines expected it to?
In an article published November 18, 2013 in the New York Times , two physicians testing the accuracy of the new risk calculator developed by the American College of Cardiology (ACC) and American Heart Association (AHA) found that it vastly overestimated patient risk. Drs Paul Ridker and Nancy Cook (Brigham and Women's Hospital, Boston, MA) calculated the 10-year risk of cardiovascular events in three large-scale primary prevention cohorts—the Women's Health Study (WHS), the Physicians' Health Study (PHS), and the Women's Health Initiative Observational Study (WHI-OS)—and found the new algorithm overestimated the risk by 75% to 150%.
The new risk-assessment calculator was developed by an ACC/AHA expert panel of physicians who authored new guidelines for the management of patient risk. Departing from the older Framingham Risk Score (FRS), the new algorithm includes a patient's 10-year risk of stroke and cardiovascular disease. Based on the new cholesterol guidelines, which were also published along with the risk-assessment guidelines, individuals without evidence of cardiovascular disease or diabetes but who have LDL-cholesterol levels between 70 and 189 mg/dL and a 10-year risk of atherosclerotic cardiovascular disease >7.5% are candidates for statin therapy.
Speaking with heartwire , Ridker said he is very supportive of the new guidelines and believes they take several steps forward in the advancement of patient care. Specifically, he is pleased with the focus on statins and the ease in which they might be implemented by physicians in clinical practice. However, he said that whenever a new clinical risk-assessment tool is developed, the single most important step is to validate the risk tool in an external population—namely, how well the predicted risk compares with the actual risk needs to be examined.
When Ridker and Cook tried to test the 10-year atherosclerotic cardiovascular disease risk score in the WHS, PHS, and WHI-OS cohorts, the risk-assessment calculator significantly overestimated risk, including in patients with a 5% to 10% 10-year risk of cardiovascular disease. In fact, regardless of the 10-year risk of cardiovascular disease, the new risk calculator overestimated patient risk compared with the actual risk in all three cohorts, said Ridker.
As reported last week by heartwire , the guideline committee also attempted to validate the risk score in the Multiethnic Study of Atherosclerosis (MESA) and Reasons for Geographic and Racial Differences in Stroke (REGARDS) studies and found that it significantly overestimated risk.
"The big picture is that they should have slowed the process down," Ridker told heartwire . "Seeing that the risk model overestimated risk in two cohorts [MESA and REGARDS], in retrospect, they should have asked colleagues to externally validate the model to determine whether it overestimated risk in other cohorts."
As it stands now, the risk algorithm significantly overestimates risk in at least five contemporary patient cohorts. For one cardiologist, it is still not too late to call a time-out.
Dr Steven Nissen (Cleveland Clinic, OH) said he has long been an advocate of using intensive doses of statins in high-risk patients, so the question for him is not about the efficacy of the drugs for reducing morbidity and mortality. "But we have to treat the right patients, and the problem is that the risk calculator in the guidelines has never been previously published and therefore could not be independently verified," he told heartwire .
And this is problematic because there are some instances where the risk-assessment algorithm is not consistent with best medical practice.
"You take an older African American man, or even someone like me, and you put them in there with really good numbers, with good blood pressure and good lipid values, and they end up being high risk," said Nissen. "Age and race seem to drive it a lot. The right thing to do is to have a pause, let's put the risk calculator out there, let's get a period of public comment, and then revise. The fear I have is that if we don't do something to revise it, we will lose public support, and we need the public to be supportive because they are the ones that have to take these medications."
Risk of Cardiovascular Disease >7.5%
Published just last week, the new guidelines are a radical departure from the previous iterations, as the emphasis is now on appropriately reducing patient risk rather than treating to a specific cholesterol target, such as <70 mg/dL or <100 mg/dL.
The new risk calculator was derived from diverse cohorts such as the Framingham Heart Study (FHS), the Atherosclerosis Risk in Communities (ARIC) study, the Coronary Artery Risk Development in Young Adults (CARDIA), and the Cardiovascular Health Study (CHS), and the equations predict the future risk of cardiovascular disease and stroke.
The findings by Ridker and Cook, which will be published Tuesday, November 19, 2013 in a viewpoint in the Lancet, led to a Saturday-night meeting with the ACC/AHA guideline authors at the American Heart Association 2013 Scientific Sessions . Included in the discussions were Dr Donald Lloyd-Jones (Northwestern University Feinberg School of Medicine, Chicago, IL) and Dr David Goff (University of Colorado, Denver), the cochairs of the guidelines on the assessment of cardiovascular risk; Dr Neil Stone (Northwestern University Feinberg School of Medicine), the chair of the guideline committee that wrote the new recommendations for treating cholesterol; and Dr Sidney Smith (University of North Carolina, Chapel Hill), a member of both writing committees, among others.
Based on their late-night meeting with Ridker and Cook, however, the guideline panel members did not feel they had enough information to question the validity of their risk-assessment calculator and didn't want to raise any alarm bells prematurely. They said they always knew the risk-assessment calculator overestimated risk, as it did so when it was tested in MESA and REGARDS.
"We were charged with developing a risk-assessment platform that would help us identify those individuals who were at high enough risk to merit intensive therapy and in whom, in conjunction with the cholesterol panel, we would expect to see real benefit and minimal harm," said Lloyd-Jones. "Obviously, we're very interested in finding out new information through scientific discourse about the risk equation, but the truth is that the risk equations work exactly as we asked them to do. There is nothing wrong with these equations."
Speaking during a press conference at the AHA, one that was hastily called to discuss the Harvard researchers' findings, Lloyd-Jones said they really have had only a "10 000-foot view" of the Ridker data but stressed that the more germane question is whether or not the risk-assessment tool was developed using the right populations. To do so, they selected populations that were representative of the US population in the 1990s. "We used very contemporary data to estimate risk in very representative populations from communities all across the United States," he said.
And while the calculator might systematically overestimate risk in certain populations, particularly in higher-risk patients, Lloyd-Jones said the WHI comprises a very healthy patient group with low cardiovascular event rates who are not entirely representative of the US population. In the other studies, they too are healthy, with low rates of smoking. "It doesn't surprise me at all that our equations would overestimate risk somewhat in those groups," he added.
For Ridker, he agrees with the characterization of the external-validation patient cohorts, but this is one of the strengths, rather than a drawback. "These are contemporary patients and reflect secular improvements in health and lifestyle changes in the past 25 years," he told heartwire . As a society, there is significantly less smoking, better exercise habits, and better diets in the more up-to-date cohorts, but this is a reflection of the strides society has made and something the AHA should be celebrating."
No Stopping Now
During the press conference, which was led by AHA president Dr Mariell Jessup (University of Pennsylvania School of Medicine, Philadelphia), the physicians stressed that the guidelines do not reflexively call for statins in patients with a risk score > 7.5%. To heartwire , Jessup said the goal of the guidelines is not to get patients on statins but to reduce their risk and have them live longer. The guidelines were released as a part of a group of four sets of recommendations that also embrace advice on treating obesity and lifestyle, she added.
During the press conference, Stone said the risk calculator does factor in patient age as a risk factor for cardiovascular disease. In addition, the risk calculator was derived with African Americans and factors in the burden of cardiovascular disease in this population. However, ACC vice president Dr Kim Williams (Rush University Medical Center, Chicago, IL) said he views the guidelines and the risk calculator as a living document, one that will eventually be updated as more data are collected.
Smith agreed, saying that they "intend to move forward with the implantation of these guidelines," but if there is something that can make them better, they will do it. "If we think there are some minor changes or some alterations in therapeutic strategies, we'll do it," said Smith.
To heartwire , Ridker said that rather than use any risk-prediction algorithm for putting primary-prevention patients on a statin, clinicians would be better served if they simply asked "what works" and "in whom." There are now a sufficient number of primary-prevention clinical trials where physicians could assign patients to statin therapy based on whether benefit was observed that "type" of patient studied. This would avoid putting patients on the drug unnecessarily and avoid denying the drugs to others despite evidence from trials showing benefit.
Stone, Smith, and Lloyd-Jones report no conflicts of interest. Disclosures for the coauthors are listed in the paper. Goff reports research support from Merck. Nissen reports consulting for all the major pharmaceutical companies, including AstraZeneca, the makers of Crestor, but does not accept financial compensation.
Heartwire from Medscape © 2013 Medscape, LLC
Cite this: How Good Is the New ACC/AHA Risk Calculator? - Medscape - Nov 18, 2013.