EHR Vendors Rated on Usability Design and Testing

Ken Terry

October 30, 2015

Only a handful of electronic health record (EHR) vendors have met best practices for user-centered design (UCD) and testing of their products under a framework created by the American Medical Association (AMA) and MedStar Health's National Center for Human Factors in Healthcare. However, the Consumer Reports–like ratings of EHRs just posted by these organizations did not compare the actual usability of those systems as experienced by the end user.

The goal of the ratings, the AMA and MedStar said in a description of the framework on the National Center for Human Factors' website, "is to draw attention to the narrow focus on only eight [usability] capabilities among the dozens required by the [Office of the National Coordinator for Health IT (ONC)] for the Meaningful Use program and to the absence of best practices in the certification process."

The partners added that ONC could use their UCD framework to improve its EHR certification program.

AMA outlined its own framework for improving the usability of EHR systems last fall. In September, MedStar researchers published a study in JAMA that found that many EHR vendors had not met ONC usability testing requirements, although their software had received certification from ONC-authorized certification bodies.

In the just-released scorecard, which rated 20 products, two Allscripts EHRs (Enterprise EHR and Sunrise Acute) and one McKesson EHR (Paragon Inpatient) received perfect scores. In the tier just below that were another McKesson EHR, Athena Clinicals, Medical Information Technology, Practice Fusion, Cerner Power Chart, Meditech Magic, and Modernizing Medicine.. Epic received only nine of a possible 15 points, and eClinicalWorks trailed the pack with a rating of five.

AMA and MedStar developed their EHR User-Centered Design Evaluation Framework "by comparing ONC's UCD certification requirements with evidence-based best practices from the human factors and usability literature," according to the website. The scorecard compares each vendor's user-centered design process, testing methodology and testing results with the evidence-based guidelines.

The framework defines user-centered design as a process that "puts the needs of the user at the forefront of the design and development, resulting in a product that is more likely to meet the needs of the user."

Low Bar of Certification

The JAMA study noted that ONC requires EHR vendors to name the UCD process they used; provide the results of usability tests, including the number, clinical background, and demographics of the participants; and make the usability reports public after the product is certified. ONC has endorsed UCD guidelines from the National Institute of Standards and Technology.

Of the 50 certified vendor reports the researchers sought, nine were not publicly available. Of the remaining vendors, 14 had not stated their UCD process, 19 used an industry standard for UCD, and six used an internally developed UCD process. There was considerable variability in the number of participants enrolled in the usability tests, and about half the vendors failed to provide the required demographic details. It was not even clear whether some of them had had any physicians test their EHRs.

"Alignment with best practices for user-centered design and testing is a starting point that regulators and industry should meet and exceed," commented Raj Ratwani, scientific director of the Human Factors Center, in a news release. "The framework we developed is the first step in bringing greater transparency to the usability processes of EHR vendors."

In the same release, AMA President Steven J. Stack, MD, said, "Our goal is to shine a light on the low-bar of the certification process and how EHRs are designed and user-tested in order to drive improvements that respond to the urgent physician need for better designed EHR systems."


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: