Proposed Frameworks to Improve the Quality of Health Web Sites: Review

Cynthia Baur, PhD and Mary Jo Deering, PhD

In This Article


An analysis of the 4 frameworks makes clear how Internet technologies and the terms of debate about acceptable practices on the Internet have shifted between 1996 and 2000. In 1996, when the HON Code first appeared, it was ground-breaking to affirm offline principles in medicine and medical publishing in an online environment: professional authority for medical and health advice; sanctity of doctor-patient relationship; confidentiality of patient data; citation of sources; evidence of health benefits for treatments, products, and services; identification of responsible parties and financial interests; and separation of advertising and editorial content. A policy statement released by the International Committee of Medical Journal Editors (ICMJE) in 1997 about publishing on the Web reinforced the idea that offline standards of peer-review and clear identification of authorship could be transferred to the Web.[4] Although the frameworks released in 2000 address many of the same topics as earlier efforts, the newer frameworks have many important differences.

One main difference relates to Web site operators' recognition that users want their privacy protected. Protection of users' privacy did not appear as an item on lists of quality criteria as of mid-1998.[5] Violations of users' privacy on the Internet are now front-page news; independent studies confirm that health Web sites do not follow their posted privacy policies[6]; and separately, HHS[7] and the U.S. Federal Trade Commission (FTC)[1] have asked Congress for national privacy legislation. Protecting privacy and ensuring quality of content have emerged as the twin pillars of the quality standards movement. In response, the Healthy People 2010 objective includes privacy and confidentiality protections as one of the elements to be disclosed to users, and the Hi-Ethics, eHealth Ethics, and AMA frameworks (the most recent of the 4) all have greater specificity on privacy and confidentiality protections than the HON principles.

Given the potential for serious abuses, one area that should have received special mention in the frameworks but did not is confidentiality protections for personal health information in electronic consumer health records. Lack of information in the frameworks on the types of protections that users will have if they rely on electronic databases to maintain sensitive information over time is a major gap. Many of the health Web sites that participated in the development of the frameworks are already providing services for consumers to store and manage individually identifiable information that they want to protect and some Web site operators and information brokers may want to exploit for commercial gain. Users must be able to anticipate what will happen to their personal information when online enterprises begin to merge, consolidate, or dissolve. The integrity and portability of their personal information must be ensured. Standard-setting for consumer health records is currently being discussed in an American Society for Testing and Materials (ASTM) Subcommittee. It is not clear how this effort will be integrated with the proposed frameworks discussed in this article.

A second important difference is the emergence of consensus around a disclosure approach for consumer protection on the Web. Five of the 8 principles of the HON Code are stated in the form of a Web site's obligation to users to operate the site in a responsible, accountable manner. By contrast, the Healthy People 2010 objective and the recent private-sector frameworks are stated in terms of disclosure, not specific obligations. The disclosure model rests on the general obligation to make available information that users need so they can make an informed decision about the quality of the site. The HON approach relies on the behind-the-scenes, professional judgments of those who design and operate Web sites and their commitment to ethical behavior (the authority of the professional), while the disclosure approach requires Web sites to provide as much information as possible about their operations to users and then lets the users decide if their personal preferences agree sufficiently with a site's policies to continue to use the sites (the authority of the consumer).

There are important limits to both approaches that should be acknowledged. With the HON approach, without additional disclosure, it would be difficult for a user to learn enough about a site's operations to know whether it adheres to avowed principles. On the other hand, the disclosure approach carries a risk of information overload and consumer confusion. The analysis highlights the sheer number of elements that an informed user might need to know (and the frameworks do not include nearly everything that a user could know about a Web site) to evaluate a site's operations. Collectively, the frameworks propose over 100 separate elements that users should read about and understand to be an informed user of a site. Although no one framework has 100 elements, many elements may appear in more than one framework and have different meanings in each one. Multiple frameworks imply that users will have to familiarize themselves with each one and understand the differences among them, as well as the differences in meaning of the terms used. Some of the information that is disclosed may be difficult to understand or subtle in its implications. For example, even if sponsorship is disclosed, will consumers be able to link this information with particular content and assess its significance?

Key elements of the 4 frameworks are discussed below.

a. HON

The HON Code predates the other 3 frameworks and represents the first attempt to provide general principles for a Code of Conduct for health Web sites of all types. It was developed by the Health on the Net Foundation, a not-for-profit organization in Geneva, Switzerland. The HON Code represents one way to set high standards, because most of the Code's principles are stated in the form of obligations and/or directives to Web site operators. Creating an obligation to behave in an ethical fashion is different from and arguably sets a higher ethical standard than creating an obligation to disclose. Without the requirement of disclosure, the HON Code leaves many pieces of information unknown to users, who may find it difficult to learn how a site claiming to adhere to the Code operationalizes the obligation to respect the confidentiality of data or the qualifications of health professionals who give advice on a Web site.

b. Hi-Ethics

The Hi-Ethics Principles are attuned to the current state of mass-market health information Web sites. The Principles represent the consensus of a group of mostly for-profit enterprises based in the United States, and were drafted with the assistance of the Washington, D.C. law firm of Hogan and Hartson. The Principles reflect the group's understanding of the emerging regulatory approach of the federal government, particularly the FTC, to the conduct of online business. The Principles also reflect the group's attempt to prove the viability of industry self-regulation in lieu of federal legislative remedies. The FTC has established its authority in online consumer protection and currently is overseeing the key areas of privacy protections and deceptive trade practices on commercial Web sites. The FTC has promoted Fair Information Practice Principles as the current standard for privacy protection and advocates industry adherence to these Principles.[1] The U.S. Food and Drug Administration (FDA), the Department of Justice, states' Attorneys General and state licensing boards also have roles in online regulation and law enforcement, but, to date, except for activities to assure the legitimacy of online pharmacies, the profiles of the other authorities have been much lower in the health sector than the FTC's.[8]

The Hi-Ethics Principles reflect the FTC's interest in information practices and nondeceptive content in several ways. The Principles appear to incorporate elements of Fair Information Practices, but the Principles use multiple terms to describe different protections, which makes it difficult to conclude that the Principles comply fully with Fair Information Practice Principles. Given the amount of commercial activity on Hi-Ethics member sites, it is surprising that the Principles do not explicitly address ecommerce as a separate category of activities. Instead, ecommerce activities are covered by principles and commitments concerning "third parties," which include commercial and noncommercial entities. The lack of specificity for what users can expect when they engage in commercial transactions on Hi-Ethics sites is a potential gap. The statement that sites will provide "reasonable support" for therapeutic claims and not provide deliberately false or misleading information; the commitment to clear separation of advertising and editorial content; and adherence to federal and state laws regarding promotional offers, rebates, and free items or services are additional examples of how the Hi-Ethics Principles conform with FTC guidance. Notably, the Principles also commit sites to disclose information about the quality of self-assessment tools, a topic not addressed by any other framework, but still do not require impact evaluation of outcomes.

The Principles have the most explicit standard for financial interest and complete discussion of sponsorship issues, including the controversial practice of targeting advertising and other content based on a user's activity while on a site. The Principles use a 10% threshold to determine financial interests that must be disclosed. Web sites will have to disclose sponsors' involvement in content creation and whether the site does targeting. The amount of attention devoted to sponsorship makes clear that sponsorship will be key to Web site operations. The Principles do not indicate whether there is any type of sponsorship relationship that would be unacceptable and that would violate ethical practices, nor do they indicate why 10% is a useful threshold, rather than disclosure of any financial interest. The issue of sponsorship raises 2 questions: Is transparency of business practices sufficient, and if so, can sites make their business practices sufficiently transparent that their claims of being trustworthy, reliable sources of information are credible? Telling users about a sponsor's involvement puts the burden on users to understand the nature of a sponsor's involvement and the potential influence on the quality of the content.

c. eHealth Ethics Initiative

The eHealth Ethics Initiative Code reflects a different sense of audience and purpose than the Hi-Ethics Principles. The Initiative brings together a wide-ranging group of online health concerns, some for-profit but also nonprofit, government and academic sites. Some of the membership of Hi-Ethics overlaps with the Initiative's core group. This Code was drafted by The Hastings Center, an independent research institute on biomedical ethics in New York City. The Initiative represents itself and its Code as international in scope and utility. One consequence of this orientation is that the Code appears to have fewer features designed with US law enforcement and regulation in mind, although the Code elements appear compatible with the US context.

The Initiative's Code has several elements not addressed by the other frameworks. It includes general warnings, which are not a required element of Fair Information Practices, to users about privacy threats and protections. Adherence to Fair Information Practices requires sites to notify users of their information practices, including which information is collected, how it is collected, how it will be used, and third-party access to information, whereas the eHealth Ethics Code requires "specific affirmative consent" for sites to collect, use, or share personal data, whether for the site or a third party. "Specific affirmative consent" is the most unambiguous of the privacy protections in any of the frameworks. Other additional protections include "adopting reasonable mechanisms to trace how personal data is [sic] used," and telling "how the site stores users' personal data and for how long." The Code also reflects a broad understanding of the concept of influence. It requires Web sites to disclose their purpose and all types of influence, not just financial, that could affect a user's assessment of a site and its contents. Health professionals are also recognized as having a central role in the design and functioning of a Web site, and the Code contains 13 elements that address how health professionals should conduct themselves. The Code directs sites to include pertinent approval information for regulated products. The Code also includes directions for clear, easy-to-read, and appropriate language for the site's intended users, and timely and appropriate responses when users contact the site with a complaint.

One of the weaknesses of the eHealth Ethics Code is that it does not make clear how functions other than information provision would be covered by the Code. These additional functions, which are covered by other frameworks, include self-assessment tools, electronic mail, chat rooms, and discussion groups. For example, are all the personal data that users disclose while interacting with a Web site covered by the proposed Informed Consent and Privacy principles? As the principles read now, they are more applicable to situations in which the site seeks to collect personal data as a result of a user's visit, and may not cover other types of information that users disclose when they participate in chat rooms, ask-a-doc sessions, and the like. The Quality Principle, which stipulates that sites will "provide health information that is accurate, easy to understand and up to date," raises the same question of comprehensiveness. The Principle seems oriented toward the site's provision of information to users. How does it apply to other information that becomes part of the site through chat rooms, etc.?

A second issue for clarification in the Code relates to the matter of editorial independence. If editorial independence will only be expected for educational and scientific information, what is the standard for sponsored information? Are the directives to disclose influence and "make reasonable efforts" to police sponsors enough to protect users in light of the proliferation of commercial and sponsored content on the Web?

A third issue relates to the standard for evidence. The Code requires that information on a site be "consistent with the best available evidence" and identified as "scientific studies, expert consensus, or professional or personal experience or opinion." This standard could be either high or low depending on how it is defined and implemented. If defined to constitute extensive, ongoing review of content created by the site and on linked sites to determine "best available evidence," then it would set a very high bar for quality. The standard would be low if it allows evidence based on very limited or poor quality science. For example, many treatments or medications may have limited evidence available at a particular moment, which makes that information the "best available" even though it may not be reliable or valid. In addition, medical researchers, clinicians, and public health officials may not agree that a particular body of information constitutes "best available evidence."

d. AMA

The AMA Guidelines, drafted by AMA staff and approved by the Executive Committee of the AMA Board of Trustees, reflect the Association's roots in medical publishing and its expansion into ecommerce and the commercial uses of information. The introduction to the Guidelines indicates that they are an update and consolidation of other AMA policies and statements about Web-based activities. Notably, the Guidelines are the only framework to acknowledge the likelihood that an increasing number of sites may place restrictions (eg, registration, passwords, subscriptions, pay-per-view) on users' access to content. It is also the only framework to disavow explicitly the navigational tricks that can direct users to specific content, such as advertisements. The Guidelines affirm the need to respect copyright and the importance of the review process to assure the quality of content, although peer review is not the only method prescribed. Information about the editorial process and content review method will be disclosed to users. Along with the Initiative's Code, it acknowledges the problem with literacy levels and the frequent ill fit between intended audiences and a lot of health information. It states, "Language complexity of the content should be appropriate for the site's audience" and, along with the HON Code, calls for clarity of writing and presentation. These directives on writing may seem minor in relation to other content criteria, but they are important in recognizing and supporting the Web as a communication medium.

Beyond the standard disclosures of financial interests for recognized authors, the Guidelines direct anyone who participates in a site, such as users who post comments in online discussion groups, to disclose their interests, if any. Like the other frameworks, the Guidelines confirm the need to keep advertising and editorial content separated. The Guidelines further commit that "advertising cannot influence editorial decisions or editorial content." Sponsorship of content is allowed if it is disclosed. The Guidelines do not specifically state that the editorial process, which is a larger domain than any individual piece of content, may not be influenced by sponsors. If the editorial process will not be influenced by sponsorship, the Guidelines should state this clearly.

The Guidelines' privacy protections seem geared to current practices on many commercial Web sites. They clearly state that AMA Web sites will collect and use "clickstream" data. The Guidelines commit Web sites to have a privacy policy with links from the home page or navigational bar; the "opportunity [for visitors] to opt in or out of allowing personal information to be tracked;" and "express permission" or "express consent" (the Guidelines are not clear if these are the same) for a site to collect, save, or share personal information. Further, it appears that users will be able to disable the cookie function and continue to use AMA sites. The Guidelines direct healthcare professionals and moderators of online discussion groups, chat rooms, or elists to follow confidentiality procedures and to be as clear as possible in helping users understand that sensitive information may be revealed when they participate in online communication. Notably, the Guidelines do not provide further direction on the participation of health professionals who provide online advice and services. If AMA-affiliated sites expand their activities in online advice and services, users may need greater protections than are currently available.