For "Junk Science" Read: "I Don't Like the Results" -- A Statistical Take on the ARBs and Cancer Study

Andrew J. Vickers, PhD


August 13, 2010

A group of researchers from a well-respected academic medical center propose a reasonable hypothesis and then test that hypothesis using a quite conventional meta-analysis of randomized trials. They come to a cautious conclusion, in particular, stating that "our findings warrant further investigation," and publish in a reputable journal.[1] Yet they are described as conducting "junk science": their "very bad paper" is said to be "irresponsible" and "dangerous," and a journalist asking for comments about the paper is greeted with expletives and dark mutterings about the paper being "planted" by competitors. Science itself is on trial: "nobody can explain why the [journal] accepted" the paper, although perhaps one might blame editorial boards being "bombarded with so-called meta-analysis," such that the system "for catching and refusing noise and rubbish occasionally leaks."

It isn't hard to work out what is going on here: it wasn't that the critics didn't like the methods section, they didn't like the results. The paper found that angiotensin-receptor blockers (ARBs) were associated with cancer. The backlash has come from those who know, like, and use ARBs. The accusations of "junk science" and of dangerous irresponsibility came from a past president of the American Society of Hypertension.[2] The conference at which expletives were used to describe the "very bad paper" was the European Meeting on Hypertension 2010.[3]

What are some of the substantive arguments given against the paper?

ARBs don't cause cancer in animal models. This might lead us to be cautious about the results but would not invalidate the paper unless we knew that animal models of carcinogenesis were infallible.

No individual trial was positive. We wouldn't expect individual trials to be sufficiently powered to detect rare adverse events, which is exactly why meta-analysis is such a valuable statistical technique.

Individual patient data would be preferable, but no individual patient data are available. This is a generic argument: there is a better way of answering the question, but the appropriate study is impossible, so we'll never really know. Such an argument could be used to deny the link between smoking and lung cancer (which depends on nonrandomized evidence) or to discount most studies on surgery (which cannot be blinded).

There was an increase in lung cancer but not in other cancers; such inconsistency casts doubt on the association between ARBs and cancer. All effect estimates were in the same direction and within the 95% confidence interval of the main analysis. Subgroup analyses are of low statistical power, but a nonsignificant P value does not mean "no effect."

Patients might misinterpret the results; the press reported the data irresponsibly. The authors expressly state that the increase in cancer risk was "modest" and of "unknown" clinical significance. At no point do the authors make any clinical recommendations, such as for patients to stop ARBs or switch to another class of drugs. If authors are appropriately cautious in their conclusions, they cannot possibly be held responsible for misinterpretation by patients or the press.

It pretty much goes without saying that our journals are awash with poorly done studies. One of my favorite statistics is that nearly 80% of the statistical tests reported in 2 American anesthesia journals were erroneously applied.[4] So it is particularly distressing that methodologically robust articles are attacked so violently. To restate, this was a meta-analysis of randomized controlled trials.

Censure can all too easily mutate into censorship. Let's have a civilized debate about the merits of papers and keep accusations of junk to studies with junk methods, not discomfiting results.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: