Scope, Completeness, and Accuracy of Drug Information in Wikipedia

Kevin A Clauson, PharmD; Hyla H Polen, PharmD; Maged N Kamel Boulos, PhD; Joan H Dzenowagis, PhD

Disclosures

The Annals of Pharmacotherapy. 2008;42(12):1814-1821. 

In This Article

Abstract and Introduction

Abstract

Background: With the advent of Web 2.0 technologies, user-edited online resources such as Wikipedia are increasingly tapped for information. However, there is little research on the quality of health information found in Wikipedia.
Objective: To compare the scope, completeness, and accuracy of drug information in Wikipedia with that of a free, online, traditionally edited database (Medscape Drug Reference [MDR]).
Methods: Wikipedia and MDR were assessed on 8 categories of drug information. Questions were constructed and answers were verified with authoritative resources. Wikipedia and MDR were evaluated according to scope (breadth of coverage) and completeness. Accuracy was tracked by factual errors and errors of omission. Descriptive statistics were used to summarize the components. Fisher's exact test was used to compare scope and paired Student's t-test was used to compare current results in Wikipedia with entries 90 days prior to the current access.
Results: Wikipedia was able to answer significantly fewer drug information questions (40.0%) compared with MDR (82.5%; p <0.001). Wikipedia performed poorly regarding information on dosing, with a score of 0% versus the MDR score of 90.0%. Answers found in Wikipedia were 76.0% complete, while MDR provided answers that were 95.5% complete; overall, Wikipedia answers were less complete than those in Medscape (p <0.001). No factual errors were found in Wikipedia, whereas 4 answers in Medscape conflicted with the answer key; errors of omission were higher in Wikipedia (n = 48) than in MDR (n = 14). There was a marked improvement in Wikipedia over time, as current entries were superior to those 90 days prior (p = 0.024).
Conclusions: Wikipedia has a more narrow scope, is less complete, and has more errors of omission than the comparator database. Wikipedia may be a useful point of engagement for consumers, but is not authoritative and should only be a supplemental source of drug information.

Introduction

Of the 80% of adults in the US who use the Internet to look for health-related information, 37% focus their search on prescription or over-the counter (OTC) drugs.[1] A number of studies have characterized the manner in which consumers search the Internet for health-related information.[2,3,4] One study in particular demonstrated that consumers search the Internet using suboptimal techniques, with one participant reporting that he thought the first 10 results returned were the only ones available.[4] Since Google is the search engine of choice for many consumers,[3] it is not uncommon for a Wikipedia entry to be the first returned result (eg, http://en.wikipedia.org/wiki/Rifampicin, as of November 5, 2008); this underscores the fact that Wikipedia is a frequently accessed resource by consumers seeking health information on the Web. Due to a "perfect storm" combining the rise of Web 2.0 technologies and a desire by patients to take a more active role in managing their health, user-edited online resources such as Wikipedia are increasingly being tapped as informational resources.[5]

Wikipedia is an extensive (>10 million articles), multilingual (253 languages), free online resource (www.wikipedia.org). As a user-edited site, it allows anyone to write, edit, discuss, and track changes to an article. In the Wikipedia entry for "Wikipedia," it states that the database "is a project that attempts to summarize all human knowledge," and the accumulation of that knowledge is done by tapping into the global network of the Internet.[6] Wikipedia is one of the most popular destinations on the Internet and recently was ranked as the ninth most visited Web site of the top 1,000,000 tracked by Compete.com.[7] Based on the frequency with which consumers are purposefully turning to or unintentionally reaching Wikipedia as a source of drug information, healthcare practitioners (ie, physicians, pharmacists, nurses) need to consider the impact that this resource could have on the behavior of and interactions with their patients. This is particularly true with respect to drug information, as it has been demonstrated that when information about drugs is in written form, it can impact patients' attitudes and behaviors.[3,8] Healthcare practitioners should be prepared to counsel patients about reputable sources for medical and pharmaceutical information on the Internet and would be well served to have some insight as to the suitability of this frequently consulted source.

Moreover, consultation of Wikipedia for drug information is not limited to the general public. Alarmingly, its content is being cited as an authoritative source more frequently by students in the healthcare professions.[9,10] Wikipedia links have also recently been included in the abstract of an article in the International Journal of Cardiology and thus similarly appear in MEDLINE.[11] Concerns raised by Taylor-Mendes[9] about using Wikipedia as a reference reflect the fact that: (1) many articles are written anonymously, (2) many contributors are students still studying the very topics about which they write, (3) some Wikipedia authors self-identify as "pre-hospital care workers," and (4) anyone can purposefully add misinformation (eg, political humorist Stephen Colbert prompted viewers to change a Wikipedia entry on elephants to incorrectly read that their population in Africa had tripled over the last 6 months).[12] Perhaps even more telling is that Wikipedia founder Jimmy Wales has stated that students and researchers should not cite Wikipedia as a source because of the high error rate.[13] The issue of entry modification with more serious consequences has come under scrutiny as well, as evidenced by the implication that several pharmaceutical companies have edited their own drug entries in Wikipedia to minimize associated safety issues.[14] Despite all of these concerns, conceptual support for the Wikipedia model exists in certain corners.[15,16]

The question remains: Is Wikipedia a dangerous source, an inconsistent source, an authoritative source, or, as other studies have found, a mostly accurate but incomplete source? The first study that attempted to use peer review to address this question compared the science content for 42 entries between Wikipedia and Encyclopedia Britannica.[17] The study found an average of 4 inaccuracies in Wikipedia per entry versus an average of 3 in Encyclopedia Britannica. The most problematic Wikipedia entry contained 19 inaccuracies of varying significance. Another study compared 25 biographical entries in Wikipedia against Microsoft's Encarta and the American National Biography Online (ANBO), which is produced by professional historians.[18] On average, ANBO was the most complete, followed by Wikipedia, then Encarta. The investigator was also surprised to find only 4 "clear-cut, factual" errors in the 25 Wikipedia entries reviewed. In one of the few attempts to evaluate the quality of health information on Wikipedia, Nicholson[19] undertook a similar effort in 2006. The study found that Wikipedia fared poorly with regard to completeness, but that 100% of the information elements covered by Wikipedia were deemed to be either mostly or entirely correct. Nicholson's recommendation in light of Wikipedia's low completeness scores was that "consumers should use this information source with caution." Finally, a study by Devgan et al.,[20] published in 2007, examined Wikipedia's entries for the most commonly performed inpatient procedures in the US and similarly concluded that, "Wikipedia is an accurate though often incomplete medical reference." They also found a correlation between an entry's quality and how often it was edited.

No study to date has attempted to evaluate the scope, completeness, and accuracy of Wikipedia for content that one-third of all Internet health-seekers search for: information on medications. Additionally, no previous study has attempted to determine whether any measured differences were statistically significant. In this study, we aimed to assess the scope, completeness, and accuracy of drug information contained in Wikipedia.

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as:

processing....