Surgeon Scorecard Stirs Debate

Lara C. Pullen, PhD

September 09, 2015

Surgery report cards were originally intended to improve quality. Instead, they may be motivating surgeons to avoid the sickest patients, which is the easiest way for a surgeon to improve his or her "grade," Lisa Rosenbaum, MD, argues in a perspectives article published online September 2 in the New England Journal of Medicine.

Despite the drawbacks, scoring efforts are here to stay. Moreover, they are being developed in an environment in which the perception is that the medical community is not doing enough to police itself.

"Consumers have a tremendous thirst for information that will help them choose the right medical provider," explained Raz Winiarsky, MD, chairman of orthopedics at Interfaith Medical Center in New York City, in an interview with Medscape Medical News. "People want to know what they are getting," he added. "When it comes to doctors, this appears to be a struggle."

In her perspective article, Dr Rosenbaum, from Brigham and Women's Hospital in Boston, Massachusetts, describes the launch of ProPublica's surgeon scorecard.

As reported previously by Medscape Medical News, ProPublica released its scorecard in July 2014, and many medical professionals responded swiftly with criticism, complaining about the quality and quantity of the data used to rank surgeons.

However, when contacted by Medscape Medical News, ProPublica defends its efforts: "[W]e have gone to some lengths to be transparent about our methods. We are also listening carefully to the critics and intend to publish Surgeon Scorecard 2.0 in the coming months," explained Stephen Engelberg, editor-in-chief at ProPublica, in an email to Medscape Medical News.

Dr Rosenbaum described in her article how some in the medical community were also turned off by the promotional video that accompanied ProPublica's scorecard. The video seemed, to them, melodramatic and sensational, adding an emotional response to something that they would prefer to see handled logically.

Despite the critical comments, Dr Rosenbaum noted in an interview with Medscape Medical News that ProPublica’s effort has prompted a more nuanced discussion on quality and how best to rank it.

For example, Dr Rosenbaum said, many in the medical community questioned ProPublica’s focus on outcomes (ie, complication rate). They also questioned whether the scorecard was capable of accomplishing what Dr Rosenbaum described in her article as the goal of "rooting out bad doctors." Instead, critics have suggested that the scorecard may have the unintended consequence of driving surgeons away from the most medically fragile patients.

Conversation Evolves

The conversation thus appears to be evolving from one that is centered on what ProPublica has done wrong to one that looks forward to determining how the next ranking — and there will inevitably be one — can be done right.

In her article, Dr Rosenbaum suggested that claims data are misleading and, perhaps, best avoided. Others have suggested that in the case of surgeons, pictures are more important than numbers, with videos of surgeries being a possible way to evaluate quality.

Most experts agree, however, that, as part of this transparency, future efforts should include a complete and simple explanation of the methodology used to create the ranking, as well as the limitations inherent in the methodology. To that end, ProPublica has described its methodology in a 21-page white paper that includes a paragraph dedicated to the limitations of their surgical scorecard approach.

Looking Forward

"The best thing that could come out of this is a lot of experts talking about it and deciding what we can do better," explained Dr Rosenbaum in an interview with Medscape Medical News.

The ultimate goal would be to capture and share with the public an insight into the practice of medicine that physicians, as insiders, already have.

For example, many in the medical profession will automatically pick up the telephone to call a friend or colleague to identify the best surgeon in a given hospital for a particular procedure. The telephone call likely also includes a request to identify the surgeon who should be avoided. In the best of all worlds, such insight will be available to all patients.

Any future efforts to accomplish this goal must include an acknowledgement that the concept of quality is complex and open for interpretation.

Dr Winiarsky, however, thinks that is missing the point. "Every provider claims to possess 'quality,' but since everyone defines it differently, it ends up being profoundly confusing, and ultimately of little help to patients. In the end, patients really only care about getting better, which involves being diagnosed correctly and getting the most appropriate care and they need tools to identify those providers that have a higher likelihood of doing that for them," he said emphatically. Dr Winiarsky is cofounder and chief medical officer of Spreemo, which provides information on price and accuracy data on radiological imaging for the employers of injured workers and insurers.

Dr Winiarsky is cofounder and chief medical officer of Spreemo. Engelberg is editor-in-chief at ProPublica. Dr Rosenbaum has disclosed no relevant financial relationships.

N Engl J Med. Published online September 2, 2015. Full text


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.