AI System Accurately Detects 53 Kinds of Eye Disease

Artificial intelligence (AI) is at least as effective as experts, and sometimes more effective, in detecting 53 kinds of sight-threatening retinal diseases, according to a paper published online August 13 in Nature Medicine.

Using Google's DeepMind deep-learning technology, the computer system made the diagnoses in seconds and had an accuracy rate of 94%, Jeffrey De Fauw, a research engineer with DeepMind in London, United Kingdom, and colleagues report. The technology is designed to help physicians prioritize which patients need care first so they can intervene before damage is irreversible.

The findings are "jaw-dropping," Pearse Keane, MD, an ophthalmologist from Moorfields Eye Hospital and University College London's Institute of Ophthalmology, London, United Kingdom, and a coauthor on the study, told London's Evening Standard .

"They are pretty stunning results and could, I think, transform the whole speciality of ophthalmology in the next few years," Keane told the newspaper.

The Evening Standard reported that the researchers hope to start clinical trials for the technology next year.

The findings "couldn't be more exciting," said Michael Abramoff, MD, PhD, from the University of Iowa, Iowa City, who was not part of the study.

However, Abramoff, who holds a patent on an AI system approved by the US Food and Drug Administration (FDA) for diagnosing diabetic retinopathy, says that autonomous AI systems that work in place of a physician, such as the one he developed, may do more to transform the specialty than assistive devices such as the one reported by De Fauw and colleagues because they address access and affordability barriers in healthcare.

Referral Accuracy Matches Specialists

De Fauw and colleagues trained the AI algorithm to spot 10 features of eye disease including lesions, hemorrhages, and fluid build-up using only 14,884 deidentified three-dimensional, high-definition optical coherence tomography (OCT) retinal scans. (The relatively small number of images used is significant, because previous work with AI has relied on databases of millions of images.)

The researchers then gathered 997 additional scans from Moorfields patients and asked the computer and eight of the hospital's consultant ophthalmologists and specialist optometrists to recommend urgent referral, semiurgent referral, routine referral, or observation for each scan.

In the most crucial category (urgent referral), the technology "matched our two best retina specialists and had a significantly higher performance than the other two retinal specialists and all four optometrists when they used only the OCT scans to make their referral suggestion," the authors write.

There were no cases in which a case of disease required urgent referral and was missed by the system, the authors write.

Computer Error Rate Comparable to Top Specialists'

When the researchers considered all referral types, they found that the computer error rate (5.5%) was comparable to the error rate for the two best retina specialists in the group (6.7% and 6.8%) and was significantly better than the other 6 experts if only OCT scans were available (10.0% - 24.1%).

When the clinicians had access to the OCT scans, fundus images, and patient summary notes, five had similar accuracy to the AI system, whereas the system still outperformed the other three.

The authors note that a key advantage of their system over previous ones is that it doesn't rely on a single device. "Our work removes previous barriers to wider clinical use without prohibitive training data requirements across multiple pathologies in a real-world setting," the authors write.

The researchers tested the AI algorithm on real-world images from 32 Moorfields sites, which cover diverse groups in London and the surrounding areas. They used 37 OCT scanners, which included two very different types of devices routinely used at Moorfields. Testing on more than one type of OCT scanner will help ensure that the technology will be applicable as scanners are upgraded over time, the authors say.

Computer Gives Reason for Decision

Another plus of the new platform is that it lets eye experts know the reason for choosing the kind of referral it suggests and provides a representation of the problem, a feature that has often been missing in previous AI programs. The authors note that this feature could be particularly valuable in difficult or ambiguous cases.

Although researchers studied only OCT scanners in this work, "future work can address a much wider range of medical imaging techniques, and incorporate clinical diagnoses and tissue types well outside the immediate application that was demonstrated here," the authors write.

Autonomous AI Will Be What Really Moves the Healthcare Needle, Inventor Says

University of Iowa's Abramoff is a retina specialist and inventor of a technology known as IDx-DR, which is the only AI device authorized by the FDA for the autonomous detection of a sight-threatening disease, diabetic retinopathy.

The device's algorithm makes the diagnosis on the basis of imaging of the patient's retina without the need for an eye specialist to interpret results.

Abramoff's device was approved by the FDA in April, and in June University of Iowa Health Care was the first medical center in the nation to start using the technology.

With regard to the De Fauw et al paper, Abramoff told Medscape Medical News, "It couldn't be more exciting because it's a very big population that they used, from Moorfields. It's also solidifying what we have been trying to do."

But he says the problem with the assisted AI described in this paper, as opposed to autonomous AI, is that assisted AI doesn't address healthcare's affordability and accessibility problems: you still need to involve the specialist. However, he said, assisted AI still has a role, and he uses it in his own practice as well.

Abramoff said, "Autonomous AI may have more potential to move the needle."

One of the advantages of the work in the current study is the relatively small number of images used, he said, noting that, "I've being doing this for almost 30 years, and we've always been faced with sparse data. Any time you can improve your algorithmic approaches to deal with that is good news."

The solution described in the current paper also includes explanations from the computer on the reasons for its referral decision. In contrast, unexplainable algorithms "fail in very unexpected ways," he said. "If you use an explainable system like this, you prevent those catastrophic failures."

Abramoff said it's important to remember that there are many more steps before clinicians can start using the DeepMind technology. His own process to provide the safety assurances demanded by the FDA took 8 years, he said.

Still to be resolved in the DeepMind work, he said, are image quality issues, who decides what images should be put in the AI, how it interacts with workflow, and regulatory aspects.

Several coauthors are paid contractors of DeepMind. Coauthors also report serving on advisory boards or accepting speaker or consulting fees from Heidelberg Engineering, Topcon, Haag-Streit, Allergan, Novartis, Bayer, Genentech, GlaxoSmithKline, Roche, Aerie, Alcon, Belkin Laser, Santen and Optos. Abramoff is founder and president of IDx-DR, a privately held company, and holds patents on the IDx-DR technology.

Nat Med. Published online August 13, 2018. Abstract

For more news, join us on Facebook and Twitter

Comments

Commenting is limited to medical professionals. To comment please Log-in.

Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.