Deep Learning Applications in Ophthalmology

Ehsan Rahimy


Curr Opin Ophthalmol. 2018;29(3):254-260. 

In This Article

Understanding Deep Learning

As a result of the surging popularity in mainstream media, the terms artificial intelligence, machine learning, and deep learning have been used interchangeably at times as synonyms; however, it is important to differentiate and distinguish the three. At the core, these can each be viewed as concentric circles, with the largest circle being artificial intelligence, and the smallest being machine learning.

Artificial intelligence is the broadest term, applying to development of computer systems able to perform tasks by mimicking human intelligence, such as visual perception, decision-making, and voice recognition. John McCarthy, widely regarded as one of the founders of artificial intelligence, defined it as 'the science and engineering of making intelligent machines'.[7]

Machine learning refers to a subfield under the umbrella of artificial intelligence, which enables computers to improve at tasks with experience, or in other words, learn on their own. One of the pioneers within machine learning, Arthur Samuel, defined machine learning as a 'field of study that gives computers the ability to learn without being explicitly programmed'.[8] That is, a machine's algorithm allows it to autonomously identify patterns in observed datasets, adjust in response to the data, and predict outcomes without having explicit preprogrammed rules and models (i.e. if-then rules).

Finally, deep learning refers to a subset of machine learning, composed of algorithms that use a cascade of multilayered artificial neural networks for feature extraction and transformation.[9,10] Drawing inspiration from the structure of the human mind, convolutional neural networks consist of thousands of individual neurons capable of performing complex tasks, such as image recognition and classification, based on pixel or voxel intensity. Each successive layer in the network uses the output from the previous layer as input, with the final layer revealing the diagnostic output. Training this type of a network requires repeatedly adjusting the parameters, known as weights, of the connections based on many teaching examples through a process called backpropagation. The network repeats this process over and over, until the diagnostic output ultimately agrees with a reference standard (i.e. what human graders assigned as ground truth). Use of the term deep, refers to the number of layers in a neural network, which contain multiple 'hidden layers' of nodes between input and output nodes. Deep learning, therefore, can be regarded as an improvement on conventional artificial neural networks by creating networks with multiple layers. Learning in this format can be classified as either supervised (i.e. classification-based) or unsupervised (pattern analysis-based). The latter represents one of the more fascinating aspects of deep learning, where large datasets are analyzed to discover underlying patterns without the need for feature engineering. Clinically speaking, instead of researchers' hand-coding instructions to an algorithm on what a microaneurysm, hemorrhage, or neovascular frond may look like on a diabetic fundus photograph, rather, they input an image labeled as 'severe nonproliferative diabetic retinopathy,' for example, and with enough labeled data, the computer eventually learns what that is. In order to train itself, a deep learning neural network is dependent upon having a variable and large enough dataset available. In the context of ophthalmology, this would require access to tens of thousands of images from a diverse patient demographic (age, sex, and ethnicity) generated through various acquisition protocols (multiple clinical sites, different camera types, mydriatic/nonmydriatic image capture). Although it is entirely possible that the algorithm independently appreciates the same classical features of diabetic retinopathy, it is also feasible that it has identified its own pattern recognition of disease beyond the scope of how humans interpret and analyze the disease, hence the 'black box' of deep learning. Elucidating what exactly the algorithm interprets is the subject of ongoing research.