Recognizing Basal Cell Carcinoma on Smartphone-captured Digital Histopathology Images With a Deep Neural Network

Y.Q. Jiang; J.H. Xiong; H.Y. Li; X.H. Yang; W.T. Yu; M. Gao; X. Zhao; Y.P. Ma; W. Zhang; Y.F. Guan; H. Gu; J.F. Sun


The British Journal of Dermatology. 2020;182(3):754-762. 

In This Article

Abstract and Introduction


Background: Pioneering effort has been made to facilitate the recognition of pathology in malignancies based on whole-slide images (WSIs) through deep learning approaches. It remains unclear whether we can accurately detect and locate basal cell carcinoma (BCC) using smartphone-captured images.

Objectives: To develop deep neural network frameworks for accurate BCC recognition and segmentation based on smartphone-captured microscopic ocular images (MOIs).

Methods: We collected a total of 8046 MOIs, 6610 of which had binary classification labels and the other 1436 had pixelwise annotations. Meanwhile, 128 WSIs were collected for comparison. Two deep learning frameworks were created. The 'cascade' framework had a classification model for identifying hard cases (images with low prediction confidence) and a segmentation model for further in-depth analysis of the hard cases. The 'segmentation' framework directly segmented and classified all images. Sensitivity, specificity and area under the curve (AUC) were used to evaluate the overall performance of BCC recognition.

Results: The MOI- and WSI-based models achieved comparable AUCs around 0·95. The 'cascade' framework achieved 0·93 sensitivity and 0·91 specificity. The 'segmentation' framework was more accurate but required more computational resources, achieving 0·97 sensitivity, 0·94 specificity and 0·987 AUC. The runtime of the 'segmentation' framework was 15·3 ± 3·9 s per image, whereas the 'cascade' framework took 4·1 ± 1·4 s. Additionally, the 'segmentation' framework achieved 0·863 mean intersection over union.

Conclusions: Based on the accessible MOIs via smartphone photography, we developed two deep learning frameworks for recognizing BCC pathology with high sensitivity and specificity. This work opens a new avenue for automatic BCC diagnosis in different clinical scenarios.


Basal cell carcinoma (BCC) is the most common skin cancer, with a rapidly rising incidence.[1] The diagnosis of BCC is straightforward for experienced pathologists but labour intensive due to the large number of cases, especially when a consecutive slide reading is needed in Mohs surgery.[2] Furthermore, the diagnosis is challenging when BCC areas vary in architecture and size, or become obscured due to inflammation. Therefore, a fast and rigorous computational method for automatic BCC diagnosis is needed.

Computer-aided diagnosis of diseases has been developed for decades to assist the analysis of microscopic images in pathology.[3,4] Most previous work has focused on 'feature engineering', which required hand-crafted features specified by domain experts.[5,6] Recent progress in deep convolutional neural networks (CNNs) has shown significantly improved performance on a wide range of computer vision tasks, including image recognition, face recognition, object detection and semantic segmentation.[7–11] Deep learning-based solutions have also demonstrated promising results on pathological image-related tasks. The CAMELYON challenge is an academic competition in recognizing and localizing breast cancer metastases in whole-slide images (WSIs).[12] Wang et al. first demonstrated the power of deep learning in breast cancer pathological diagnoses.[13] The group of Norouzi, who ranked first in the challenge leaderboard in 2018, pointed out that cancer metastases could be detected from whole-slide pathology images through some carefully designed networks.[14] These previous studies focused on WSIs and leveraged classification on image patches for detecting and localizing metastases.

In this study, microscopic ocular images (MOIs) photographed from microscope eyepieces using smartphone cameras were used to develop neural network models for recognizing BCC automatically. We first investigated a CNN classification model, which did not perform well in detecting BCC in hard cases (see details in the Materials and methods). We further found that the hard cases could be well recognized by pixelwise segmentation methods. Hence, a cascade framework was created, with an initial CNN classification model and a subsequent semantic segmentation model. Alternatively, we created another framework of the semantic segmentation model only. This segmentation framework achieved better performance on BCC recognition, yet required more computational resources. Further analysis demonstrated that the performances in recognizing BCC were comparable based on WSIs or MOIs. This indicates that recognizing BCC through a smartphone could be considered a future clinical choice, especially in some screening cases.