We Made a Map; It's Probably Wrong

Brenda Goodman and Andy Miller

June 28, 2021

Editor's note: This story explains the data behind a new Medscape report, Disturbing Number of Hospital Workers Still Unvaccinated.

Using federal data, Medscape and WebMD made a map of the percentage of healthcare workers at hospitals around the US who have not yet been vaccinated against COVID. Seems simple enough. It isn't.

Anyone should be able to look at this map we created and learn what percentage of healthcare workers at their local hospital have not yet been vaccinated against COVID-19.

That may not be possible, however, because some hospitals say the data are incorrect. The information was directly sourced from the US Department of Health and Human Services (HHS), which compiled its numbers from hospital surveys. So how could the numbers be wrong? The answer to that question is complicated.

In the end, though, the shoddy data mean neither the government nor consumers have enough information about vaccination coverage in the nation's hospitals, where surprising pockets of hesitancy exist.

Prior to COVID, the government collected some health information from hospitals, but that data were primarily based upon insurance billing. And it was historical information, released months or even years after the fact.

When the pandemic arrived in the United States, policymakers needed a snapshot of conditions inside hospitals in real time, but no system had been set up to collect information on things such as hospital capacity, staffing levels, and equipment shortages.

In March 2020, former Vice President Mike Pence, who was then head of the White House Coronavirus Task Force, sent a letter to hospital administrators across the country asking for daily reports.

The government devised a survey of 32 questions — it has since been expanded — that hospitals are supposed to answer each day. The questions ask about things such as the number of staffed intensive care unit beds, ventilators, the number of patients who tested positive for COVID-19, and the number of deaths from COVID-19.

It's a wide-ranging survey that requires hospitals to compile information from different internal departments. Many hospitals, which had already been struggling to cope with increased patient loads, found the new data collection onerous.

Flaws and Inconsistencies

There were several false starts. The American Hospital Association said the administration threatened last June to send National Guard troops into hospitals to collect the statistics.

Then, in July 2020, Deborah Birx, MD, who was then coordinator for the White House response, wrested control of the data collection system that was maintained by the Centers for Disease Control and Prevention (CDC) and gave it to a politically connected company called Teletracking Technologies.

In August 2020, the government made this reporting one of the conditions of participation in the Medicare and Medicaid programs, meaning that if hospitals fall behind or fail to share their data, they don't get reimbursed for their care. Hospitals get millions from Medicare reimbursements annually.

Teletracking Technologies began collecting information from hospitals and reporting it to the federal government, which kept the information largely under wraps. It wasn't publicly released until the Government Accounting Office made a series of critical recommendations urging HHS to make its data collection more transparent.

Some of the data are now on public view, but they're not always useful because of all the flaws and inconsistencies.

Some Hospital Numbers Are Correct

On the basis of spot checks by Medscape Medical News 664199892 and Georgia Health News, some hospitals have reported their data correctly.

AdventHealth, which has 49 hospitals, is one of the largest health systems in the United States. At least 20 of those hospitals report that fewer than half their workers have been vaccinated against COVID.

Jeff Grainger, a spokeperson for AdventHealth, confirmed that the vaccination numbers reported to HHS were correct but said they probably underestimate the true number of workers who have gotten their shots, because the hospital system can't account for cases in which workers go off campus to be inoculated, such as to a local pharmacy or mass vaccination site.

AdventHealth Orlando, which is the fifth largest hospital in the United States, according to Becker's Hospital Review, reported that 56% of workers were not vaccinated as of the week of June 4, 2021.

The hospital said it has surveyed its employees about taking the COVID vaccines and has used that information to shape messaging, but it has not taken other steps ― such as offering bonuses for participation in the vaccination campaign, as some other healthcare systems have.

Grainger said they require employees to get flu vaccines and may do the same for COVID vaccines after the vaccines have been fully licensed by the US Food and Drug Administration.

AdventHealth is affiliated with the Seventh Day Adventist church, though the church does not own or operate the hospitals, Grainger said in a statement.

The Seventh Day Adventist church has issued a statement in support of "responsible immunization/vaccination."

Confusion Pervades Hospital Data Reporting

Other hospitals have misunderstood how to report.

According to HHS data, only 4% of the nearly 12,000 people who work with patients at Grady Memorial Hospital, in Atlanta, Georgia, are unvaccinated. That figure includes staff and contractors. But the hospital said that number is wrong. Instead, 45% of workers at Atlanta's public hospital, a busy level I trauma center, are still unvaccinated.

Grady said that they accidentally included in the running total of vaccinated workers at their facility any healthcare worker they had vaccinated — even those who didn't work at their hospital.

On the other end of the spectrum is Geisinger Health System, in Pennsylvania. Federal data show that at their hospital in Danville, 98% of workers are unvaccinated. The hospital said that's incorrect — they estimate that 28% of employees at that campus remain unvaccinated.

Geisinger, like many hospitals across the country, reports its numbers to its state health department, which in turn sends the data to the federal government.

The federal government allowed state health departments or hospital associations to become certified to report the information on behalf of their hospitals.

When pressed, a Geisinger spokesperson said she couldn't explain why the numbers were wrong. She suggested we contact HHS and the State Health Department of Health for more answers.

Amber Liggett, a spokesperson for the Pennsylvania Department of Health, confirmed that they have been collecting information daily from the state's hospitals and had been passing it to the federal government.

Health departments are useful conduits because the data allow them to see where problems may lurk at the state and local level. But the Pennsylvania Department of Health isn't using the vaccination data because it is so flawed.

"We don't know what data verification is happening at either the facility or federal level; and there currently isn't any at the state level," Liggett said in an email.

Another common mistake has to do with the idea of running totals.

Hospitals were asked to report the vaccination data once a week, on Wednesdays.

The numbers are meant to be cumulative. If a hospital inoculated five workers in its first week and then another five its second week, the totals reported for weeks 1 and 2 should be 5 and 10, respectively, according to this explanation by the CDC.

Some hospitals, such as CalvertHealth, in Prince Frederick, Maryland, are reporting their numbers week by week and are relying on the government to add them up. Using the numbers in the example above, the hospital's totals for week 1 and week 2 would be 5 and 5.

In addition, CalvertHealth is moving vaccinated workers into the unvaccinated column if they didn't get their shots that week.

Let's say a nurse named Fred works at CalvertHealth and that Fred got his second shot of the Pfizer vaccine on Friday, June 4. The next Wednesday, the hospital would report him among their fully vaccinated workers for that week. The following week, though, Fred would be moved to a column tracking unvaccinated workers because, the hospital told us, he wasn't among the workers vaccinated that week.

As a result, it looks as uf CalverHealth's numbers of unvaccinated workers are growing week over week while their number of fully vaccinated workers is getting smaller because they've hit a saturation point with vaccinations at their hospital.

When we run our calculation, it looks as if nearly all of CalvertHealth's employees are unvaccinated, but the hospital tells us that's incorrect — about 30% still have not gotten their shots.

States Step In

In other cases, the data are being collected, but states are blocking the release of the information.

We contacted Houston Methodist Hospital, in Houston, Texas, to find out why its numbers are not in the database. The hospital's CEO has been outspoken on the issue of worker vaccinations and has been publicly announcing the hospital's progress.

Houston Methodist was confused too. They insisted that they were reporting the numbers.

It turns out that although Houston Methodist is sending the data to the Texas Department of State Health Services (DSHS), the state is blocking the release of the numbers.

In fact, only 4% of Texas hospitals have numbers in the system.

Chris Van Deusen, a spokesperson for DSHS, said the agency was withholding the numbers because they were voluntary fields, "and the confusing and overlapping nature of the questions could lead to skewed data," he said.

In Nevada, only 2% of hospitals are reporting the vaccination status of their workers.

Amy Shogren, a spokesperson for the Nevada Hospital Association, admitted that they are not collecting vaccination data from the state's hospitals for the government's HHS database because Nevada's hospitals report that information to a different database that is designed to collect information on vaccine administration. She declined to say why the numbers couldn't be reported in both systems.

Keeping Workers and Patients Safe

Vaccination of healthcare workers matters because it protects these frontline professionals who work in occupations that often put them in contact with the SARS-CoV-2 virus. More than 3600 healthcare workers have died over the course of the pandemic in the United States, according to a recent investigation by Kaiser Health News and The Guardian.

It is also a patient safety issue. People who catch COVID spread the virus before they show any symptoms. Thus, healthcare workers could pass the virus to vulnerable patients before they knew to stay home from work, especially if they work in a setting in which there is no regular testing program.

Although someone who is healthy might not experience a severe SARS-CoV-2 infection, a patient whose immune system is weakened by medication or a health condition could die.

Houston Methodist Hospital, in Texas, recently announced that nearly 100% of its healthcare workers were vaccinated. Houston Methodist is the first hospital in the nation to mandate the vaccines. Other hospitals have followed suit, bostered by a ruling from the Equal Opportunity Employment Commission that employers could require the vaccines as long as they offered reasonable exemptions for workers.

But there are still many hospitals in the United States — particularly in rural areas — where large portions of the staff are still unvaccinated. And there's no way for patients to find out before they walk in.

Letitia Armstrong recently took her husband to Baptist Health, in Lexington, Kentucky, where she reassured the male nurse who was treating him that both she and her husband were fully vaccinated, adding "I'm sure you've had your vaccine, too."

He said he had not. When she asked why, he said, "Well, I'm young, and I figure if I get it, I wouldn't get very sick."

Armstrong, who runs an eldercare company in the area, said she was furious. Because her husband was very ill and the nurse was working on him, she held her tongue.

Baptist Health isn't reporting data on vaccinations of its healthcare workers, and it doesn't have to. Reporting is currently voluntary.

About 48% of hospitals across the country either have declined to reveal the vaccination status of their workforce to the federal database or their state isn't providing it.

Patient safety experts think the staff vaccination numbers should be required, too.

Reporting of employee vaccination rates "should be mandatory so patients can make a decision [as to which hospitals to go to] if they have a choice,'' said Mark Ebell, MD, professor of epidemiology and biostatistics at the University of Georgia, in Athens.

"For someone who is immunocompromised in a hospital, the last thing they need is to get a serious infection," he said.

Fixing the Data

As in many other areas of public health, decades of underfunding and complacency have weakened America's public health data collection. The pandemic blew holes right through it.

"We weren't ready for it," said Charles Rothwell, former director of the National Center for Health Statistics, who retired in January 2019 after three decades at the agency.

Although Rothwell and others sounded warnings that America's public health data systems needed more investment, he said he feels personally responsible for not making a compelling enough case to policymakers. "We should have been better prepared," he said.

Even as the United States took weeks to develop and then scale an effective test for the virus, public health agencies were struggling to compile even the most basic numbers on the health emergency.

Journalists and data experts at nonprofit organizations jumped in to fill the gaps with invaluable and heroic efforts, such as the COVID tracking project and COVID Exit Strategy. The COVID tracking project, started by journalists at The Atlantic, used a network of hundreds of volunteers to compile daily numbers on cases, hospitalizations, tests, and deaths. COVID Exit Strategy, which evaluated trends in daily cases and tests, was helmed by a group of public health and data experts who worked for nonprofits and universities.

"We're talking about basic data about how many people are in the hospital. What are they in for? How many people died? And how much protective equipment do you have?” said Jennifer Madans, who recently served as acting director of the National Center for Health Statistics.

She said the United States has whole agencies with expertise in how to collect and report information such as this, but they were not tapped for help.

Instead, the contract was given to Teletracking. It has been providing data to the HHS Protect Dataset, which is the data we used to make our map.

The data are now on public view, but they're not always useful because of all the flaws and inconsistencies.

That's not surprising to Madans, who said that historically, the United States has spent more money and resources on tracking agriculture statistics than it spends compiling basic health information.

In March 2020, the government allocated $500 million to the CDC through the CARES Act to upgrade its data collection.

But experts say more money is needed to help our health data infrastructure catch up, and efforts are underway to improve the system.

A bipartisan group of lawmakers introduced the Health Statistics Act in February to fix information gaps and barriers that were exposed by the pandemic. The act would allocate another $450 million to the system, as well as standardize health data collection and require the government to share it more broadly across agencies.

It would also require the participation of agencies such as the National Center for Health Statistics (NCHS) that have expertise in this kind of data collection.

Jennifer Madans said the investment is long overdue and has never been more urgent.

She said if NCHS had created a database such as the one we used to make our map and that it was found to be so full of flaws, "we would have been just destroyed. It would have been terrible."

Brenda Goodman is a senior news writer for WebMD and Medscape. Andy Miller is editor and CEO of Georgia Health News. Chris Bolton contributed additional reporting. Follow Brenda Goodman on Twitter: @ReporterGoodman

For more news, follow Medscape on Facebook, Twitter, Instagram, and YouTube.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: