Little Evidence for or Against Pay-for-Performance Plans

September 07, 2011

September 7, 2011 — As part of healthcare reform, the federal government is testing different ways to reward physicians for the quality — not quantity — of their services, all in the hope of improving patient care.

A new Cochrane review published today underscores the importance of conducting trial runs for quality-oriented payment models, such as accountable care organizations and medical homes. The authors found insufficient evidence in the medical literature to support or oppose these kinds of incentive schemes.

"Implementation [of such programs] should proceed with caution," write lead author and economist Anthony Scott, PhD, a professor at the University of Melbourne in Melbourne, Australia, and coauthors.

Their article appears in the current issue of the Cochrane Library, an online collection of databases created by the Cochrane Collaboration, a nonprofit international organization that evaluates medical research.

The lack of evidence cited by the authors should not suggest a lack of effort both here and in other countries to reward clinicians for quality of care. Numerous US private insurers have operated "pay for performance" programs over the past decade.

However, the authors of the Cochrane review could find only 7 studies on financial incentives deemed to have a satisfactory design. They limited themselves to randomized controlled trials, controlled before-and-after studies, and interrupted time-series analyses, all of which had to evaluate the effect of financial incentives on the work of primary care physicians. The selection criteria defined quality of care as patient-reported outcome measures, such as overall satisfaction, clinical behaviors (eg, test-ordering), and clinical and physiologic measures (eg, blood pressure and cholesterol levels).

Of the 7 studies, 5 examined incentive plans in the United States, and 2 looked at counterparts in Germany and the United Kingdom. All of the US plans were concocted by private insurers to coax quality improvement out of large medical practices. In one example, a clinic received a $5000 bonus if it referred at least 50 smokers to a telephone counseling service, and $25 for each additional referral beyond that. In another plan, the insurer withheld a percentage of each group’s fee-for-service revenue to fund an incentive pool. The pool money was distributed to the groups according to their performance in the realms of clinical quality, patient satisfaction, and practice efficiency. Higher-ranking groups received up to 150% of their withheld revenue, while the lowest ranking group recouped only half, thus incurring a penalty.

Studies Faulted for Not Addressing Selection Bias

Six of the 7 studies in the review showed "positive but modest effects on a minority of the measures of quality of care included in [each] study," the authors write. This faint praise was further weakened by design flaws identified in the Cochrane review. For example, performance bonuses paid by private insurers in 6 of the studies went to the group practice, not individual physicians, and these studies did not report how the groups distributed or otherwise used the bonus money.

"If tasks are delegated to a physician assistant or nurse, then the issue is raised as to whether they were paid a share of the financial reward," the authors write, noting that who receives what has a bearing on effective teamwork.

A more serious design flaw, the authors point out, is the failure of all of the studies to address selection bias — that is, how did physicians come to participate in financial incentive programs?

"Physicians who provide poor quality of care may withdraw from the health plans providing these schemes, choose to contract with health plans that do not have incentive schemes, or choose not to participate in the study, thus leaving the ‘better performing’ physicians in the study," the authors explain. "Observed improvements in performance may therefore be due to selection rather than an actual change in physician behavior."

Future studies of pay-for-performance programs, the authors conclude, should be more carefully designed to address selection bias and other issues with study design.

The literature review was supported by the Australian Primary Health Care Research Institute and the University of Melbourne. The authors have disclosed no relevant financial relationships.

Cochrane Database Syst Rev. Published online September 7, 2011. Full text.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.