Clinicians Are Still Not Satisfied With EHRs

Raj Ratwani, PhD

Clinician satisfaction with electronic health record (EHR) usability has not improved, according to the findings of a new study which compared system usability scale scores from 2014—2015.

Raj Ratwani, PhD, director of MedStar Health National Center for Human Factors in Healthcare, and his colleague analyzed 27 EHR vendors that used the usability scale and found that system usability scale scores decreased for 44% of vendors, from 2014—2015.

“So, what we’re seeing here is that despite the focus on the importance of EHR usability and satisfaction, things are actually not improving when we look at those scores,” Ratwani told MD Magazine®.

The US Department of Health and Human Services (HHS) Office of the National Coordinator for Health Information Technology requires that all vendors test the usability of their products for certain capabilities, including satisfaction. A common way to measure satisfaction is through the system usability scale score, which is a 10-item survey tool that ranges from 0—100.

A system usability scale score of 68 was considered the average benchmark, while a score of 80 was considered above average, based on an analysis of 200 studies of various products in various industries.

Ratwani and colleague Kylie Gomes, MS, a PhD candidate in the department of engineering systems and environment at the University of Virginia, looked at the scores reported by vendors for their 2014 and 2015 certifications to better understand whether usability was improving over time.

“What we found was that there was actually no statistical increase in the (system usability scale) scores, representing no statistical increase in usability and satisfaction over time,” Ratwani said.

The system usability scale scores for 2014 and 2015 products were not statistically different (73.2 [16.6] vs 75.0 [14.2]; t₂₆ = .674; P = .51). In 2014, 9 (33%) products were below the average benchmark score of 68; 18 (67%) were at or above average; and 11 (41%) met or exceeded the above average benchmark score of 80. In 2015, 7 (26%) products were below the average benchmark; 20 (74%) were at or above average; and 12 (44%) met or exceeded the above average benchmark.

During the 2-year period, system usability scale scores for 12 products (44%) decreased, 13 (48%) increased, and 2 (7%) were unchanged.

Clinicians reported that if the EHR doesn’t meet their workflow and when there are extended documentation requirements, they are more likely to be dissatisfied with the technology.

The implications of an EHR system that is not easy to use and that clinicians are not satisfied with could be major. For one, difficult EHR systems could lead to clinician burnout, or even a professional leaving the medical field. Clinicians who are burnt out are also more likely to commit suicide or commit an error—which impacts patient safety.

Although vendors are working to improve EHRs, all stakeholders need to come together to make the technology better. It is important for vendors to improve their user-centered design processes and their usability testing to better understand specifically which components of the technology contribute to the challenge of satisfaction.

“That requires human factors and usability and this approach of really getting to know the clinical users well and understanding that all the different specialties in medicine are going to have different thoughts about what should be in this product and what’s going to improve satisfaction,” Ratwani said.

Vendors need to increase the amount of research they’re doing, and clinicians need to be willing to participate in the research and be vocal about their needs and issues with the technology.

If clinicians at a healthcare facility are not satisfied with their product, increasingly, the organization will switch their EHR system. But it becomes tricky to do that because the cost of the products can be high and because it is difficult to rapidly move all of the patients to the new product.

Previous research by Edward Melnick, MD, from the department of emergency medicine at Yale University School of Medicine, and colleagues looked at satisfaction scores from implemented EHRs in the context of burnout, rather than satisfaction scores from the certified products and certification testing.

Investigators found that the average score reported by clinicians on their implemented EHR products was 45.

“This points to this major reality gap,” Ratwani said. “We need to find a way to overcome this because what’s happening is that the scores from certified products are quite higher than what we’re seeing in terms of scores from implemented products, and that difference comes from how much your product changes during that implementation.”

The study, “Evaluating Improvements and Shortcomings in Clinician Satisfaction With Electronic Health Record Usability,” was published online on JAMA Network Open.