MOC Tied to Better Physician Performance Scores in ABIM Study

Marcia Frellick

June 11, 2018

Maintenance of Certification (MOC) was linked to better physician performance scores on some measures in a study conducted by the American Board of Internal Medicine (ABIM) and published online today in the Annals of Internal Medicine.

Critics, however, say the differences are small and lack clinical meaning, and they question the researchers' methodology.

Researchers compared performance on six Healthcare Effectiveness Data and Information Set (HEDIS) process measures for physicians who did or did not maintain certification 20 years after initial certification. Of the 1260 physicians in the study, 786 (62.4%) maintained their certification from 1991 to 2012 and 474 (37.6%) did not.

Researchers looked at Medicare claims data for 85,931 patients and calculated physicians' performance on the basis of the percentage of patients who received the appropriate tests.

They found that regression-adjusted differences in performance scores between those who maintained their certification and those who did not were positive and statistically significant for 4 of the 5 individual measures examined and for a composite score based on 3 diabetes measures.

The largest statistically significant difference was observed for patients receiving biennial mammograms (72.0% adherence for physicians who maintained certification vs 67.8% adherence for those who did not; regression-adjusted difference, 4.6 percentage points). The smallest significant difference was for low-density lipoprotein cholesterol testing among patients with coronary heart disease (79.4% vs 77.4%; regression-adjusted difference, 1.7 percentage points).

For the individual diabetes performance scores, the regression-adjusted differences in adherence were 4.2 percentage points for semiannual hemoglobin A1c testing (58.4% for recertified vs 54.4% for non-recertified physicians; 95% confidence interval [CI], 2.0 - 6.5 percentage points; P < .001), 2.3 percentage points for annual low-density lipoprotein cholesterol testing (83.1% vs 80.5%; 95% CI, 0.6 - 4.1 percentage points; P = .008), and 1.3 percentage points for biennial eye examinations (74.7% vs 73.4%; 95% CI, −0.8 to 3.3; (P = .112).

For the composite of the three diabetes measures, the difference was 3.1 percentage points (46.0% vs 41.6%; 95% CI, 0.5 - 5.7; P = .019).

Editorialist Questions Methodology

Critics, however, call into question both the methodology and the significance of the results.

In an accompanying editorial, Lee Goldman, MD, MPH, from Vagelos College of Physicians and Surgeons, Columbia University, New York City, said he has always been in favor of MOC, but not the way it is conducted.

Moreover, the results from Gray and colleagues have little clinical meaning, he says. Of the 6 measures analyzed, only mammography has been correlated with better outcomes. He adds that physicians other than the internists surveyed may be the ones ordering testing for the measures studied, such as endocrinologists ordering diabetic care and an ophthalmologist ordering an eye exam.

In addition, he points out the authors did not evaluate how the relative importance of MOC compares with more commonly used measures such as the physician's medical school site, residency training, and current hospital affiliations.

However, he says the most disturbing part of the study is the low bar for meeting HEDIS standards and the only slightly better performance by the MOC physicians.

"Rates below 50% for diabetes warrant a grade of F," he writes, "and there is little solace in finding a characteristic that raises a physician's grade to a slightly higher F. For [low-density lipoprotein] cholesterol testing in patients with coronary heart disease, the grade would still be C, with an adjusted difference of only 1.7 percentage points. And for mammography, it went from D+ to C−, with a difference of 4.6 percentage points that could easily be explained by differential access to gynecological co-care."

Goldman said, "[I]f [MOC] simply takes my physician from a low F to a slightly higher F, or even from a D+ to a C−, on metrics that may or may not be worth grading, then I do not find it helpful at all."

Earlier Paper Links MOC to Less Risk for Disciplinary Action

The newly published article is one of several in recent years that study links between MOC and several outcomes. An ABIM study published in March in the Journal of General Internal Medicine found MOC is associated with fewer state medical board disciplinary actions.

In that study, Furman S. McDonald, MD, from the ABIM and an adjunct professor of medicine at the Mayo Clinic in Rochester, Minnesota, and colleagues found the risk for discipline among physicians who did not pass the internal medicine MOC examination within the 10-year required period was more than twice that of those who did pass (adjusted hazard ratio [aHR], 2.09; 95% CI, 1.83 - 2.39). Disciplinary measures did not vary by state CME requirements (aHR 1.02; 95% CI, 0.94 - 1.16), but declined with higher MOC scores (Kendall's tau-b coefficient, −0.98 for trend; P < .001).

ABIM's Gray told Medscape Medical News that the two papers were among several the board conducted "to validate the way we're looking at certification. And this sort of research feeds into that," he said.

The findings are not causal, he pointed out, but they suggest associations.

"Clinical significance for these types of measures is always difficult to assess, and that is definitely one of the limitations of the study," Gray said.

Both studies point in the same direction, he said. But they were not motivated to counter the current wave of physician protest around MOC, he said, explaining that the original application to get the data from Medicare occurred before the current wave of controversy.

He said about the impetus for the series of papers: "We're not advocates for anything. In fact we're just trying to understand the facts around MOC."

"This is all part of a really broad research agenda to improve and validate what we do," he said.

Critic Calls the Studies "Propaganda"

Westby Fisher, MD, a cardiologist in Evanston, Illinois, who is affiliated with NorthShore University Health-Evanston Hospital, told Medscape Medical News the studies are "propaganda pieces."

He pointed out that both studies were produced by ABIM and researched by ABIM employees.

He added that the newer one is published in the Annals of Internal Medicine, which is the journal of the American College of Physicians, "who helped create the maintenance of certification guidelines with the American Board of Internal Medicine."

Both studies are also observational, Fisher noted, adding "they don't prove anything."

In the study by Gray et al, he said, HEDIS measures "can only tell you if a person drew a hemoglobin A1c or not twice a year. But that doesn't tell you how good the doctor is."

In light of these studies, physicians will push back, Fisher predicted. "It's really sad to see that these people will really go to these measures to put out this kind of banter to convince doctors and payers that this MOC program is legitimate. It just is not."

Editorialist Goldman said certifying bodies should have to prove the process adds "true value."

He writes, "The medical profession would be better off if certifying bodies, with their enormous prestige and moral stature, focused on how the recertification process can add the most true value to the public and their physician constituencies, rather than on trying to prove that physicians who ran the gauntlet of their initial unpopular and often burdensome dictums were statistically associated with unimpressive differences in medical practice. The recertification glass is not entirely empty, but we all wish it were more full."

The study by Gray et al was funded by the ABIM. Gray and several coauthors are employed with or are consultants for ABIM. Coauthors also report nonfinancial support from Myers-JDC-Brookdale Institute and the City of Newton, Massachusetts, and personal fees from RTI International, UpToDate, Evidera, and Freedman Healthcare Consulting outside the submitted work. McDonald and coauthors all are employed by ABIM. Goldman reports receiving personal fees from Goldman-Cecil Medicine outside the submitted work. Fisher is an unpaid board member of Practicing Physicians of America.

Ann Intern Med. Published online June 11, 2018. Article, Editorial extract

For more news, join us on Facebook and Twitter


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: