William W. Culbertson, MD


December 27, 2011

Determination of Valid Benchmarks for Outcome Indicators in Cataract Surgery: A Multicenter, Prospective Cohort Trial

Hahn U, Krummenauer F, Kölbl B, et al
Ophthalmology. 2011;118:2105-2112


Increasingly, institutions, third party payers, and the general public are requesting or requiring assessment of the quality of medical procedures and treatment. Payers want to ascertain what they are paying for and compare what they are receiving to established acceptable outcomes or normative "benchmarks." Providers likewise wish to market favorable results for competitive purposes.

Evaluations of quality might review treatment processes or quantify the outcomes of treatment. Although the concept of "quality care" seems obvious, defining what constitutes quality processes or outcomes for a given treatment is elusive and often controversial. Uncertainty exists as to what parameters to assess, how to assess them and how accurate and honest reporting can be ensured. This difficulty is compounded when a refined procedure such as cataract surgery (which has a predictably high success rate regardless of who performs it) is being scrutinized. When only a few percentage points separate the high and low end of outcomes, who is to say what is acceptable and what is substandard performance? Confounding any quality evaluation are issues of variable patient comorbidities, differing patient cultures, and outcome assessment standardization.

Study Summary

In this study of standardized cataract procedures, Hahn and colleagues quantified 2 potential benchmarks for ascertaining quality outcomes: achieved final visual acuity, and deviation of the achieved refractive outcome from the intended refractive target. In the course of their study, we find out how much seemingly simple benchmark indices can vary among study sites, with no apparent explanation.

Hahn and coauthors recruited 1685 patients to have cataract surgery in 1 eye at 7 different sites (206-239 patients per site) performed by 7 surgeons known for their excellent cataract surgical technique. Cataract surgery was standardized for intraocular lenses (AlconSN60AT), lens calculation formula (SRK/T) and A constant (118.7). Patients with other ocular disorders that could affect the outcome were excluded. Interestingly, eyes with age-related macular degeneration who had a visual potential of better than 0.2 were included. Eyes were assessed at 1 day, 1 month, and 3 months following surgery. Results showed the absolute variation of the target refraction to within ≤ 0.5 diopters ranged from 70% to 85% at 1 month and from 70% to 94% at 3 months. The percentages of achieved best corrected visual acuity ≥ 0.8 ranged from 74%- 89% at 1 month and from 81%-93% at 3 months. The investigators could not explain the variability of results among theoretically comparable patients, surgeons, technique, and surgical facilities. At 3 months, 98.5% achieved best corrected visual acuity (BCVA) of 0.5. In comparison with other less rigorously conducted benchmarking-type studies reviewed in the paper, this 98.5% BCVA of ≥ 0.5 exceeded reported rates varying from 94.6% to 95.4% for eyes without comorbidities. In the study patients, the overall average relative frequency of deviation of the postoperative refraction of ≤ 0.5 diopters was 80.3% whereas in comparable studies, the average deviation ranged from 44.6% to 65.7%. Thus the target refraction and visual acuity endpoints were significantly better in this study than similar studies, again without an obvious explanation. The investigators proposed "benchmarks for outcome indicators of cataract surgery" of 80% ≤ 0.5 diopters residual refractive error and 87% ≥ 0.8% BCVA. Given these recommended achieved percentages, 3 of 7 of the study sites and surgeons would fall below these arbitrarily selected "benchmarks" for both residual refractive error and BCVA.


Given the apparent inexplicable differences in results among matched sites, surgeons, and eyes in this carefully conducted study, it is difficult to conceive how more heterogenous groups could be reasonably compared. The complexities of sorting out the potentially confounding variables and reporting inconsistencies inherent in even this most basic analysis strains the credibility of any standard benchmark application for assessing quality. Other quality parameters that assess complications, such as capsular breaks, cystoid macular edema, and corneal edema, or quantifiable parameters, such as phaco energy, surgery time, endothelial cell loss, and the like, depend on documentation and accurate reporting. Thus the laudable goal of assessing and establishing benchmarks for "quality cataract surgery" appears to be still a relatively elusive and hypothetical goal.