Warning! Media Distorts Results With Alarming Headlines!

Henry R. Black, MD; Andrew J. Vickers, DPhil


May 19, 2011

This feature requires the newest version of Flash. You can download it here.

Henry R. Black, MD: Hi. I'm Dr. Henry Black, Clinical Professor of Internal Medicine at the New York University School of Medicine and immediate past President of the American Society of Hypertension. I'm here with my friend and colleague Dr. Andrew Vickers from Sloan-Kettering talking about biostatistics in the 21st century. My concern about a lot of modern statistics, at least a modern interpretation, meta-analysis being 1 example and database analysis being another, is that the headline writer just writes something that [he or she] hopes will get people to read the article.

Andrew J. Vickers, DPhil: Right.

Dr. Black: If you hear, for example, that a particular drug causes cancer when that is not really what was said, or the data were not great, some people will look at this and say, "Well, I'd better stop taking that drug." Or a doctor says, "I'd better not give that drug." There is a large impact from the kinds of statistics we now have to use. We don't have very many big, easily interpretable clinical trials that point to something. We now have to use other methods to get there. That concerns me and I don't know what we can do about that.

Dr. Vickers: It's a big problem. I actually call it the "toilet problem." I call it the toilet problem because when I was renovating my house, we had to buy a toilet. We went into the Home Depot® or whatever it was and we said, "Can we buy a toilet?" They said, "Which one do you want?" I said, "What do you mean? There are different ones?" They started telling us there's a low-flush and high-flush and this one and that one. Toilets are really complicated and you had to be an expert in toilets to be able to choose one.

Then take medical research, which is several orders of magnitude more complicated than indoor plumbing. We have all these sophisticated statistical methods such as imputation and intention-to-treat. We have to sort of boil all that down to a simple message. It's actually very difficult to do. You have to be very sophisticated as a journalist to do that. There aren't many journalists who can do it. What I often find when following something on Google® is just variations on the press release.

Dr. Black: Even if they are sophisticated, and even if the article is excellent, the headline writer might not be sophisticated and people don't read much beyond the headlines or even in between.

Dr. Vickers: No. The headline is often something like "Cure Offers Hope to Sufferers," and that's it. It's a big problem -- just how sophisticated medical research has become. That is because we are looking for ever smaller facts. In your own field, for example, I remember when I was growing up, if you had a heart attack, you were going to die. That was pretty much it.

Dr. Black: Yes, it was pretty easy to count bodies.

Dr. Vickers: Now if you have a heart attack, you're going to survive it. We're trying to actually reduce even further the relatively small number of people who are dying. So with the smaller facts we're going to have to use more sophisticated methods, and they are increasingly difficult to understand, even for a clinical researcher as sophisticated as yourself.

Dr. Black: I'm kind of curious. Sometimes I read (this is primarily in the lipid literature), people seem to think they can reduce the risk for heart attack to zero. They reduced it to 33% from 80%. What about the last 33%? That implies that we can get it to zero, and [that is] not possible. Things are going to happen no matter how many statins we have or drugs we add to statins or what else we have.

There is a misinterpretation, and I have a lot of pity for the patients who are reading these articles. The doctors who are reading them have to contend with the patient who says, "Don't give me that drug; I heard it was bad for me." "Well, you're not 42 years old with asthma and living in Singapore. This is you."

Doctors don't have the time or the sophistication to understand the difference. They probably never read the study's inclusions and exclusions. We accept clinical trials that have a really select cohort. I don't want to call them populations -- they are never populations. They are select cohorts that don't necessarily represent the whole society.

No hypertension trial that I am aware of includes [patients with] insulin-dependent diabetes. Yet we see the results and hear about how good a particular drug is and that a [person with] insulin-dependent diabetes will be receiving it. Yet, effectiveness in that population wasn't shown in the trial.

Dr. Vickers, thank you very much for your input. It is so valuable right now that we understand these things when we are trying to figure out what to do with a patient, what to tell a patient, and what a patient is supposed to interpret from what they read. I appreciate your time.

Dr. Vickers: Thank you. It was fun to be here.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: