Cancer Center’s Top Ranking Aided by ‘Massive’ Error

Mortality data was incorrect

Nick Mulcahy

August 28, 2013

The University of Texas MD Anderson Cancer Center, in Houston, has been the top-ranked oncology hospital in the US News & World Report's annual evaluation for the past 7 years.

However, the first-place finish was enhanced in each of those years by a huge data error by the center, according to a report last month in the Cancer Letter, a trade publication.

MD Anderson misclassified its admissions from the emergency department, which count for approximately 40% of the annual admissions at the center. The miscoding of these patients, who typically have higher mortality than other oncology admissions, effectively erased them from rankings-related data. This resulted in a lower mortality ratio, which is one of the most important variables used by the magazine in its rankings.

The repeated error, which has now been corrected, may or may not have affected the top spot in the list over the last 7 years, said Avery Comarow, the health rankings editor at US News.

"Is it possible they could have flipped with Memorial Sloan Kettering? It's conceivable, but I would guess unlikely," Comarow told Medscape Medical News about the 2 top finishers in each of the last 7 years, including 2013. "It certainly would have reduced the gap," he added, referring to, for example, the 100 scored by the Texas center and the 93 scored by the New York City center this year.

The data error would not have affected other individual centers' rank, Comarow also explained.

"M.D. Anderson said they were not aware of it and I believe them," said Comarow about the error, and also pointed out that the misclassification was MD Anderson's fault.

The discovery of the mistake was actually prompted by Memorial Sloan-Kettering Cancer Center, the perennial runner-up in recent years' rankings.

"Going back several years, some of my MSK colleagues became puzzled by the difference between our mortality data and MD Anderson's. We attempted to replicate the data, and communicated with US News…in an effort to understand the discrepancies," said Andrea Molinatti, a spokesperson for Memorial Sloan-Kettering, in an email to Medscape Medical News.

The rankings will not be changed to reflect the discovered error because "no reliable way could be found to adjust the results to reflect the missing data," according to the Cancer Letter article.

In an interview with the newsletter, Comarow said, "we couldn't retroactively rerun the cancer rankings." He also admitted that some percentage of the data, which is taken directly from Medicare information, will be "screwed up." However, the MD Anderson data was "massively" in error, he said.

The US News rankings are taken "very, very seriously" by MD Anderson, according to historian James Olson, PhD, author of Making Cancer History, an institutional history of the cancer hospital, who spoke with Medscape Medical News last year.

Dr. Olson, a professor of history at Sam Houston State University, in Huntsville, Texas, said that the center has had a long-held ambition to be preeminent. One of the former top administrators,Lee Clark, MD,"thought about being the top cancer center in America every day" and was "quite calculated" in pursuing that goal, said Dr. Olson. "M.D. Anderson has cultivated its reputation," he added, especially among physicians.

The payoffs of a top ranking include various institutional benefits, such as fundraising leverage and attracting patients locally and globally, points out the Cancer Letter.

The marketing power of the US News rankings can be found when looking at any American metropolitan area with a competitive hospital market: the rankings are touted and the US News emblem is prominently displayed online and in print advertising, billboards, and various other ad venues.

Elements of a Ranking

The US News best hospitals rankings, including those of cancer centers, are based on a 4-pronged scoring system.

Mortality, which is adjusted for morbidity, accounts for 32.5% of the score. The other variables are: patient safety (5%), reputation (based on a survey of specialists, 32.5%), and other care measures such as technology and patient/nurse ratios (30%).

At MD Anderson, patients who were admitted to the hospital through the emergency department were incorrectly coded as transfer patients since 2004, according to an MD Anderson internal document obtained by the Cancer Letter.

However, because transfer patients are excluded from the US News rankings, these patients were not counted into the data, which, in turn, affected mortality results.

MD Anderson admitted as much in the internal document. "Mortality rate higher for admitted Emergency Center patients (which were inappropriately excluded) leading to a lower mortality ratio," reads a PowerPoint presentation dated November 2012, written by Thomas Feeley, MD, head of the division of Anesthesiology and Critical Care, and Victoria Jordan, PhD, MBA, director of Quality Measurement and Engineering.

The data source for the rankings is the Medicare Provider Analysis and Review (MEDPAR), the CMS database for Medicare hospital patients, explained Comarow. US News does not receive mortality data directly from hospitals, he said.

The error was caught in 2009 and subsequently corrected. However, because the US News' system averages hospital data over a number of years, this year's top ranking was still partially affected by the data error.

MD Anderson did not respond to a request for information for this article.

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as:

processing....