Readers' and Author's Responses to "Are Traditional Peer-Reviewed Medical Articles Obsolete?"

Michael J. Goodman, PhD; Dean Giustini; Robert A. Goldstone, MD; Jerry Ann Ward, PhD; Peter Frishauf


March 17, 2006

To the Editor,

If all that you say were true, Wikipedia would be a win.[1] Unfortunately, in medicine this is not always the case. After I read your article, I went to Wikipedia to look up something where the consensus of the evidence is clear, so it should be simple for Wikipedia, vaccine safety.

Instead of what you suggest, for example, I found that the Wikipedia article on the Vaccine Safety Datalink (VSD) is composed almost entirely of attacks from autism advocacy groups of the VSD data-sharing agreement. The only medical article that is cited is a widely discredited article by Mark and David Geier.[2] (See, for example, the Institute of Medicine report that specifically questions its conclusions.[3]) The consensus of peer-reviewed literature flatly contradicts every point that is made in this article.[4]

As far as balance, there is not a single reference to any study ever performed by the VSD. Without beating a dead horse, there are at least 18 peer-reviewed publications that speak specifically to critical issues of vaccine safety, none of which are cited in the Wikipedia article.[5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22] If this is the unbiased view that Wikipedia will bring, I guess I'll wait for peer review.

For something like Wikipedia to function as you suggest requires constant vigilance. But only the advocates have the time to police it as required for it to be unbiased.

If this article is a representative example, I dread what medical treatment would look like without peer review.


Michael J. Goodman, PhD
Senior Research Investigator
Minneapolis, Minnesota


  1. Frishauf P. Are traditional peer-reviewed medical articles obsolete? MedGenMed. 2006;8:5. Available at: Accessed January 6, 2006.

  2. Geier MR, Geier DA. Thimerosal in childhood vaccines, neurodevelopment disorders, and heart disease in the United States. J Am Physicians Surg. 2003;8:6-11.

  3. Institute of Medicine. Board on Health Promotion and Disease Prevention and Institute of Medicine. Immunization Safety Review Committee. Immunization safety review: vaccines and autism. Washington, DC: National Academies Press; 2004:199.

  4. Parker SK, Schwartz B, Todd J, Pickering LK. Thimerosal-containing vaccines and autistic spectrum disorder: a critical review of published original data. Pediatrics. 2004;114:793-804.

  5. Lewis E, Shinefield HR, Woodruff BA, et al. Safety of neonatal hepatitis B vaccine administration. Pediatr Infect Dis J. 2001;20:1049-1054.

  6. Verstraeten T, Davis RL, DeStefano F, et al. Safety of thimerosal-containing vaccines: a two-phased study of computerized health maintenance organization databases. Pediatrics. 2003;112:1039-1048.

  7. Jackson LA, Austin G, Chen RT, et al. Safety and immunogenicity of varying dosages of trivalent inactivated influenza vaccine administered by needle-free jet injectors. Vaccine. 2001;19:4703-4709.

  8. Black S, Shinefield H, Ray P, et al. Risk of hospitalization because of aseptic meningitis after measles-mumps-rubella vaccination in one- to two-year-old children: an analysis of the Vaccine Safety Datalink (VSD) Project. Pediatr Infect Dis J. 1997;16:500-503.

  9. Chen RT, Pool V, Takahashi H, Weniger BG, Patel B. Combination vaccines: postlicensure safety evaluation. Clin Infect Dis. 2001;33(suppl4):S327-333.

  10. Davis RL, Marcuse E, Black S, et al. MMR2 immunization at 4 to 5 years and 10 to 12 years of age: a comparison of adverse clinical events after immunization in the Vaccine Safety Datalink project. The Vaccine Safety Datalink Team. Pediatrics. 1997;100:767-771.11. Davis RL, Kolczak M, Lewis E, et al. Active surveillance of vaccine safety: a system to detect early signs of adverse events. Epidemiology. 2005;16:336-341.

  11. Davis RL, Kramarz P, Bohlke K, et al. Measles-mumps-rubella and other measles-containing vaccines do not increase the risk for inflammatory bowel disease: a case-control study from the Vaccine Safety Datalink project. Arch Pediatr Adolesc Med. 2001;155:354-359.

  12. DeStefano F, Mullooly JP, Okoro CA, et al. Childhood vaccinations, vaccination timing, and risk of type 1 diabetes mellitus. Pediatrics. 2001;108:E112.

  13. Eriksen EM, Perlman JA, Miller A, et al. Lack of association between hepatitis B birth immunization and neonatal death: a population-based study from the vaccine safety datalink project. Pediatr Infect Dis J. 2004;23:656-662.

  14. Bohlke K, Davis RL, Marcy SM, et al. Risk of anaphylaxis after vaccination of children and adolescents. Pediatrics. 2003;112:815-820.

  15. Jackson LA, Nelson JC, Whitney CG, et al. Assessment of the safety of a third dose of pneumococcal polysaccharide vaccine in the Vaccine Safety Datalink population. Vaccine. 2006;24:151-156.

  16. Kramarz P, DeStefano F, Gargiullo PM, et al. Does influenza vaccination exacerbate asthma? Analysis of a large cohort of children with asthma. Vaccine Safety Datalink Team. Arch Fam Med. 2000;9:617-623.

  17. Niu MT, Rhodes P, Salive M, et al. Comparative safety of two recombinant hepatitis B vaccines in children: data from the Vaccine Adverse Event Reporting System (VAERS) and Vaccine Safety Datalink (VSD). J Clin Epidemiol. 1998;51:503-510.

  18. Ray P, Black S, Shinefield H, et al. Risk of chronic arthropathy among women after rubella vaccination. Vaccine Safety Datalink Team. JAMA. 1997;278:551-556.

  19. Plesner AM. Allergic reactions to Japanese encephalitis vaccine. Immunol Allergy Clin North Am. 2003;23:665-697.

  20. DeStefano F. The Vaccine Safety Datalink project. Pharmacoepidemiol Drug Saf. 2001;10:403-406.

  21. Kramarz P, DeStefano F, Gargiullo PM, et al. Influenza vaccination in children with asthma in health maintenance organizations. Vaccine Safety Datalink Team. Vaccine. 2000;18:2288-2294.

Author Replies: I agree that Wikipedia cannot and should not be relied on for accuracy -- it's open nature is simply too vulnerable to be trusted universally.

A few points:

In my conclusion to the editorial, I said, "a variant of Wikipedia for medicine is the future -- and it's good." What the time constraint of a video editorial did not allow was an explanation of what that variant may look like. I was alluding to a future publishing opportunity that would combine the strength of Wikipedia -- readers get to read a single article updated in near-real time, but with much more quality control: Every entry would be contributed and peer-reviewed by many experts selected by the publisher.

I am pleased that by pure serendipity, shortly after I recorded the editorial, just such a venture, Digital Universe, was announced. One of the founders is Larry Sanger, who also helped launch Wikipedia, but left just because he believed that there was inadequate respect for expertise in the Wikipedia method. The editors of Medscape were kind enough to allow me to add a "suggested reading" about the Digital Universe venture to the text portion of the editorial (see: Terdiman D. Wikipedia alternative aims to be 'PBS of the Web.' Available at: Accessed December 30, 2005.)

It is a worthwhile public service for experts, such as Dr. Goodman, to register as a Wikipedia user and correct misinformation in it when they encounter it. The Wikipedia "rules of engagement" are that articles be written in NPV -- neutral point of view. When they are not, Wikipedia can "protect" a page so that it can only be edited by trusted contributors. It's important to correct misinformation if for no other reason than that Wikipedia is so often consulted. Current data show that Wikipedia is visited more often than The New York Times, The Washington Post, and CNN combined. That's far more often than peer-reviewed medical Web sites.

What I find remarkable about Wikipedia is not that it contains bad information, but how much of it is actually good. As someone raised in the culture of traditional peer-reviewed publishing, I would never have expected the overall level of quality that is found in Wikipedia, as has been confirmed in the study by Nature and others, cited in the "Suggested Readings" portion of the editorial. The issue of trust is always difficult to evaluate as periodic, well-publicized slipups by even the most "trusted" primary-source periodicals (most recently with Science on human cloning). I agree with Dr. Goodman that for now, traditional peer-reviewed publishers, such as Medscape, with an openly disclosed and publisher peer-review process by experts should be the most trusted. But other systems for peer review and publishing may, in the future, prove superior. Time will tell.

Peter Frishauf


To the Editor,

Thanks to Ves Dimov at Clinical Cases and Images Blog who alerted me to the Medscape videocast "Are Traditional Peer-Reviewed Medical Articles Obsolete?[1]" [Registration (free) required.] Medscape founder Peter Frishauf says that "a variant of Wikipedia for medicine is the future for peer review in medicine.[1]"

On December 29, 2005, I wrote about wikis in the context of Internet search here as a possible solution to freeing up "locked-down" medical content, which seems more and more likely in an era of open access. Collaborative social technologies, such as blogging and wikipedias for medicine, have an undeniable appeal: Will they be short-lived or revolutionize the way that physicians communicate best evidence?

Dean Giustini

UBC Biomedical Branch Librarian
Vancouver, British Columbia, Canada


  1. Frishauf P. Are traditional peer-reviewed medical articles obsolete? MedGenMed. 2006;8:5. Available at: Accessed January 6, 2006.


To the Editor,

I could not disagree more with Peter Frishauf about the superiority of Wikipedia over peer-reviewed journals.[1] Although his criticisms of the peer review process are valid, my attempts at learning something from Wikipedia have led to misinformation. Anyone, regardless of credentials, can make entries. It is wishful thinking to believe that responsible knowledgeable experts will immediately correct factual errors. This is unfortunate, as the general public now gets much of its medical information and misinformation from the Web.

In my own field of expertise, orthopaedic surgery, I went to Wikipedia to search for "sciatica." Under the section entitled "Treatment," I found the comment that "Chiropractic manipulation often helps." But upon following the link to chiropractic from that sentence, I found the statement that "The studies found no benefit to treating chronic pain or sciatic nerve irritation."

There will always be a need for peer review, just as there is a need to be able to rapidly disseminate medical information of vital immediate importance. I submit, however, that an unsupervised, nonmedical entity, such as Wikipedia, is not the proper vehicle.

Robert A. Goldstone, MD


  1. Frishauf P. Are traditional peer-reviewed medical articles obsolete? MedGenMed. 2006;8:5. Available at: Accessed January 6, 2006.

Author Replies: Agree: Peer review is good. But traditional peer-reviewed articles may be obsolete. As stated in my reply to Dr. Goodman (above), I believe that a panel of peer reviewers and expert authors collaborating on a single topic may just be the best way to create and distribute a trusted body of current scientific information.

Peter Frishauf


To the Editor,

I suggest that you, Peter Frishauf, and others at Medscape or WebMD are the very best placed to test the hypothesis that a "Wikipedia of Medicine" is the paradigm of the future.[1]

Prepared with ready arguments, I read the Frishauf editorial on Wikipedia -- only finding that Peter had handily punctured each, some before I could even formulate them, so I say to you, "Go for it!" See how it works.

A "Medipedia" would be a very handy way to quickly build consensus on antecedents, causes, treatments, outcomes of diseases, and conditions. Imagine being able to sort out the myriad articles published on that most horrific of drugs, warfarin, for instance.

What better place to test this hypothesis than on Medscape?

Jerry Ann Ward, PhD


  1. Frishauf P. Are traditional peer-reviewed medical articles obsolete? MedGenMed. 2006;8:5. Available at: Accessed January 6, 2006.


Readers are encouraged to respond to George Lundberg, MD, Editor of MedGenMed, for the editor's eye only or for possible publication via email:


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: