Artificial Intelligence in Ophthalmology: A Brave New World Coming

Linda Brookes, MSc


August 16, 2018

Artificial intelligence (AI) has been heralded as part of "the fourth industrial revolution," along with Internet of Things, blockchain, and edge computing. All are methods of collecting, analyzing, and storing information and viewed as possessing the same life-changing potential as the introduction of electricity in the early 1900s. The integration of AI into healthcare, which has already begun, is expected to radically change clinical practice, presenting healthcare practitioners with new challenges.

Although many physicians are apprehensive about the potential impact of AI and whether it can live up to the expectations generated by recent hype, eye care specialists appear optimistic about its eventual role in their profession, looking forward to a time when their skills and expertise are augmented by AI as it streamlines diagnosis and treatment while reducing clinical errors and variability. Many expect that AI will allow them to see more patients at a time when the number of patients continues to rise as the population ages[1] and many people do not receive appropriate eye screening or eye care,[2] and when there is an existing shortage of ophthalmologists.[3]

A typical view of AI is that of Rahul Khurana, MD, from the department of ophthalmology, University of California, San Francisco, who sees it as ushering in a "brave new world" in ophthalmology.

Charles Wykoff, MD, PhD, from Houston Methodist Hospital in Texas, believes that "the potential of AI is tremendous, and it presents a great opportunity to improve the efficiency of our treatments and improve outcomes for patients."

At the same time, Khurana cautions that, "whenever you have a new technology, it's opening a Pandora's box and practicing ophthalmologists have concerns about things we don't yet know about."

This "box" was recently opened in the United States with the regulatory approval of the first autonomous AI system for the detection of diabetic retinopathy in the United States.[4]

Progress in Diabetic Retinopathy

Screening for diabetic retinopathy has been a primary focus in the development of AI systems because of the increasing number of people with diabetes (30.3 million in the United States in 2015, with an estimated diagnosis of 1.5 million new cases[5]), around 29% of whom have diabetic retinopathy.[6] Diabetes and ophthalmology guidelines recommend that people with diabetes have an annual dilated eye examination,[7,8] but only 50%-65% of people with diabetes are likely to be screened.[9]

"This is a major problem," Khurana says. "These patients are in primary care, but many are not being referred to an ophthalmologist in a timely manner and we know that when you don't treat diabetic retinopathy early, you can get irreversible vision loss. It is a public health burden and the hope is that AI technology will help identify these patients and make more efficient use of our resources. The problem for the busy ophthalmologist is that they are seeing patients who have no eye issues and that is not an efficient use of their time. If we can identify the patients who really need to be seen by a specialist, that is a more efficient use of our limited resources."

The newest AI systems for diabetic retinopathy, including the recently approved screening tool, IDx-DR (IDx, LLC; Coralville, Iowa),[4,10] involve computer software algorithms that simulate the progressive layers of neuronal functions in the human cortex (convolutional neural networks, or CNNs).[11] A CNN algorithm can be trained through structured exercises in reading patient data and medical images (deep learning) to recognize, identify, and interpret digital patterns in color fundus photographs or optical coherence tomography (OCT) scans.


The IDx-DR screening algorithm analyzes digital images taken with a nonmydriatic retinal camera and then diagnoses the presence or absence of more-than-mild diabetic retinopathy. In a validation study of 900 patients that formed the basis for its US approval, the algorithm correctly identified 87.4% of patients who had the disease (sensitivity) and 89.5% of patients who did not have the disease (specificity).[4] A notable feature of the study was the participation of novice operators with no previous experience in fundus photography and having completed only one prior 4-hour training program, suggesting that the system could be used by staff not usually involved in eye care.[12]

The first IDx-DR went into use at the diabetes and endocrinology clinic of the University of Iowa Health Care-Iowa River Landing in June; IDx expects other healthcare systems to adopt the system later in 2018.[13]

On the Horizon

Software giant Google has also developed a deep learning algorithm for detection of diabetic retinopathy initially based on fundus photography[14] and later expanded to include OCT images. The newest algorithm is being trained to diagnose and assess progression of diabetic retinopathy and neovascular (wet) age-related macular degeneration (AMD) in a collaboration between Google sister company DeepMind Health and Moorfields Eye Hospital in London, United Kingdom.[15] The training dataset has been created from over 1 million images of patients who attended Moorfields for routine scans over the past 10 years. The algorithm has shown "promising signs," and findings have been submitted to a medical journal.[16] If the results pass peer review, the technology could enter clinical trials within a few years, according to Dominic King, MBChB, PhD, DeepMind's clinical lead.

Moorfields consultant ophthalmologist Pearse Keane, MD, believes that the algorithm will help address an increasing number of false-positive referrals received by hospitals like Moorfields.[17] "Too many patients are being referred for the wrong reasons, leading to a clogging up of the clinics and the resulting inability of clinicians to treat genuine cases of sight loss like diabetic retinopathy or wet AMD within an appropriate time scale," he says.

Extending Artificial Intelligence Further

Retinopathy of Prematurity

The success in using AI and deep learning in diagnosing diabetic retinopathy has raised the possibility of exploring deep learning algorithms for use in retinopathy of prematurity (ROP).

A major challenge is that clinical ROP diagnosis is based solely on the appearance of retinal vessels on dilated ophthalmoscopic examination at the neonatal intensive care unit bedside. The decision to treat is primarily based on the presence of plus disease,[18] but diagnosis of plus disease is highly subjective and variable.[19] A CNN-based deep-learning algorithm trained to diagnose plus disease from over 5000 retinal images has been developed by researchers from Oregon Health & Science University in Portland, Oregon, and Massachusetts General Hospital in Boston. In an independent test set of 100 images, the algorithm was able to accurately diagnose the condition 91% of the time compared with a team of 8 ROP experts who achieved an accuracy rate of 82%.[20]

"There's a huge shortage of ophthalmologists who are trained and willing to diagnose ROP," said study co-lead, Michael Chiang, MD. "The algorithm allows clinicians who have little experience with ROP to help babies receive a timely, accurate diagnosis."

Macular Degeneration and Diabetic Macular Edema

Several studies have reported successful detection and categorization of AMD as early or late stage with deep learning algorithms trained using fundus photographs[21] or OCT images[22] with an accuracy comparable to that of human experts.

Recently, Kang Zhang, MD, PhD, from the University of California San Diego, La Jolla, and colleagues in the United States, Germany, and China reported the development of an AI platform that diagnoses diabetic macular edema (DME) and choroidal neovascularization seen in neovascular AMD and issues referral recommendations ("urgent" for DME or AMD, "routine referral" for drusen, and "observation only" for normal).[23] In a validation set of 1000 images (633 patients), it was able to diagnose with an accuracy comparable to that of experts and generate a decision about referral for treatment within 30 seconds with more than 95% accuracy.

Screening for Glaucoma and Beyond

Although development of AI in glaucoma has not matched the progress in retinal diseases,[6] some promising results with deep-learning algorithms have recently emerged.

The "holy grail" for AI in ocular health is a single autonomous system that can screen simultaneously for diabetic retinopathy, glaucoma, and AMD.

Visulytix, a tech company in London, has developed a retinal AI platform, Pegasus, that autonomously screens for glaucoma while simultaneously classifying the stage of diabetic retinopathy. Initial results of a study[24] carried out in collaboration with the Massachusetts Eye and Ear Infirmary in Boston were "encouraging," according to Visulytix. Using images from around 400 patients attending a tertiary care glaucoma clinic, this study suggested that Pegasus software may be more sensitive than expert evaluation.[24] Further studies are ongoing, the company says.

A deep learning algorithm that diagnoses glaucoma using fundus photography was recently reported by University of Tokyo researchers.[25] In a testing data set, sensitivity at the specificity of 95.0% was 70.3%.

An AI algorithm to detect and track a range of glaucoma indicators using OCT scans is in development with IDx. A glaucoma early-detection product, IDx-G, will "probably" go into clinical trials in 2018, according to IDx president, Michael Abramoff MD, PhD.[26]

The "holy grail" for AI in ocular health is a single autonomous system that can screen simultaneously for diabetic retinopathy, glaucoma, and AMD and could be located at optometry practices, primary care clinics, and pharmacies. The algorithm would provide diagnosis and personalized treatment recommendations based on an individual's risk for disease progression and blindness. A deep learning system that could be used as a "fully-automated" screening model to detect referable possible glaucoma and referable AMD in addition to diabetic retinopathy has been described by Singapore researchers.[27] In an evaluation of nearly half a million images from multiethnic community, population-based, and clinical datasets, the model showed high sensitivity and specificity in identifying all three diseases. Unlike other models, it was developed from a screening program that included poor-quality images from a range of different cameras.

More Data and Regulations Needed

Many healthcare professionals fear that AI may be introduced into clinical practice without sufficient testing.[28,29] They point to a lack of published peer-reviewed, prospective data supporting the new systems. Problems with algorithms, including reproducibility and interpretability (understanding the "black box"), have been called out by software engineers. AI researchers from Google and the University of California, Berkeley, recently announced that algorithms have become a form of "alchemy," with researchers not knowing why one works and another does not, and they pointed to a lack of "empirical rigor" in the field of machine learning.[30,31] The datasets required for training algorithms should consist of high-quality, labeled data; however, they can be expensive to create and can run into problems with weak labeling (tagging by experts) because of a lack of consensus among the experts providing the standards for algorithm development.[32]

Regulatory strategies and legal issues surrounding AI are still under discussion. The IDx-DR diagnostic system was evaluated by the US Food and Drug Administration (FDA) under the "De Novo" premarket review regulatory pathway for novel, low-to-moderate-risk devices.The device had been designated a "breakthrough device," allowing the FDA to guide the company in its development and to expedite evidence generation and regulatory review.

In April 2018, FDA commissioner Scott Gottlieb, MD, announced that the agency would be "implementing a new approach to the review of artificial intelligence."[33] This will employ the "Pre-Cert" approach, allowing companies to make minor changes to devices without having to make resubmissions each time.

Legal liability in cases of misdiagnosis with AI is another issue that has yet to be resolved.[34]

Incorporation Into Clinical Practice

The use of AI in healthcare calls for a culture change in how clinicians and patients entrust clinical care to machines.[29] Both physicians and patients will need to trust a "black box" to determine a disease state, but there is still debate about whether it will ever be fully accepted by physicians or patients.[6,35] With more understanding of the black box, it is likely that trust will be established, but there is also anxiety that if and when AI is accepted, patients might come to trust it more than they do physicians.

Some investigators believe that AI has the potential to extend the expertise of physicians, but there is fear that it could also cause a loss of self-confidence and affect the willingness of a physician to provide a definitive interpretation or diagnosis. There is also concern that overreliance on technology could lead to deskilling, which has been reported in other areas of medicine.[28]

...There is also anxiety that if and when AI is accepted, patients might come to trust it more than they do physicians.

AI systems that screen for early disease are likely to devolve onto primary care practices, and while this will free up time for ophthalmologists, it will create additional work for primary care providers. How this can be resolved, who pays for the new equipment, and how primary care practices are reimbursed is under discussion. Ophthalmologists appear confident that they will not be replaced by the new technology, as is frequently predicted for other specialists.[36] "I don't think AI is ever going to replace doctors. We still need people to explain these applications and give them the human context. Human contact is what patients want," Khurana says.

This is the "classic response" to be expected from specialist physicians, according to entrepreneur and venture capitalist Vinod Khosla in Menlo Park, California, who in 2016 predicted that eventually around 80% of the time physicians spend on medicine will be replaced by "smart hardware, software, and testing."[37]

Others close to AI research like Krishna Yeshwant, MD, MBA, Google Ventures, and Brigham and Women's Hospital in Boston, predict that, particularly in ophthalmology, "the world will look at it ultimately not as machine learning taking over but rather these areas finally kind of getting some of the benefits of computer science and becoming more efficient."[35]

"AI is going to bring more patients into the system," Khurana says, "and if we can identify the patients who really need to be seen by a specialist, this is a more efficient use of our limited resources."

But one problem will remain that AI cannot fix. As Paul P. Lee, MD, JD, from the University of Michigan Medical School and W.K. Kellogg Eye Center in Ann Arbor, recently pointed out;[35] although AI programs will screen more people, there is still a problem in terms of getting people into care after they've been screened. "To close the loop, we have to keep that in mind," he stressed.

Follow Medscape on Facebook, Twitter, and Instagram.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: