Blinded Comparison of Computer-Aided Detection with Human Second Reading in Screening Mammography

Dianne Georgian-Smith; Richard H. Moore; Elkan Halpern; Eren D. Yeh; Elizabeth A. Rafferty; Helen Anne D'Alessandro; Mary Staffa; Deborah A. Hall; Kathleen A. McCarthy; Daniel B. Kopans

Disclosures

Am J Roentgenol. 2007;189(5):1135-1141. 

In This Article

Results

Readers' Performances

The primary reader called back 475 of 6,381 cases (7.4%) ( Table 1 ). Biopsies were recommended in 70 of the 475 cases (14.7%) called back by the primary reader. There were 13 malignancies (18.6%) detected in these patients for a primary reader screening detection rate of 2.04 cases per 1,000 women (13/6,381), reflecting that our population is one of primarily incident cancers because 78% of the population had previous studies.

An additional 30 cases were called back by the CAD reader for a CAD additional callback rate of 0.47% (30/6,381) cases ( Table 1 ). Of these 30 callbacks, three (10%, 3/30) were recommended for biopsy. There were no malignancies.

There were 34 cases (34/6,381, 0.53%) that were called back by the second human reader that were in addition to the primary reader's callbacks ( Table 1 ). Except for one case, these cases were different from those that were called back on the basis of CAD markings. Five of the 34 cases (14.7%) were recommended for biopsy. Of the second reader's recommendations that went to biopsy, two of the five (40%) were malignant. The increase in cancer detection contributed by the second reader was from 13 by the primary reader to a total of 15 cases for a relative increase in the cancer detection rate of 15.4% (2/13) ( Table 1 ).

There was no statistical significance in the performance between the CAD and second human readers as measured by the recall rates and by the relative increase in recall rates (p =0.70) and in cancer detection rates (p =0.50) ( Table 1 ). However with a difference detected between the CAD and second human reader of only two cancers, no statistical significance could be shown due to such a small number. In contrast, the difference in the number of call-back cases between the two readers was much greater. With 63 cases of disagreement, we would have had 80% power to detect the difference if either reader was associated with twice as many callbacks as the other reader.

The overall screening cancer detection rate for all three readers was 2.35 per 1,000 (15/6,381). In addition, there were three interval malignancies not detected by any of the readers that developed within 1 year. There were, thus, a total of 18 cancers at the time of imaging in the cohort, for a prior probability of 2.82 cases per 1,000 women.

Malignancies Detected by the Second Human Reader

Two additional cancers not seen by the primary reader or the CAD reader were called back by the second reader (Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, Figure 6 and Figure 7). Of clinical significance is that the CAD system had marked both of the lesions, but the markings were dismissed by the primary reader. Both cases were called back for diagnostic workup by the second reader that resulted in recommendations for short-interval follow-up. Biopsies were subsequently recommended on the basis of findings at the 6-month workup, and malignancies were then diagnosed. Because cancer was detected within 1 year of the incident screening as a result of the second human reader's calls, these cases were counted as true-positive calls for the second reader.

Figure 1.

Malignancy detected by human second reviewer in 52-year-old woman with ductal carcinoma in situ (DCIS) who presented with group of three or four punctate calcifications on screening mammograms. At time of screening, these calcifications had arguably been stable for 3 years. Therefore, human second reviewer's motivation to recommend additional views is unknown. At diagnostic visit, radiologist thought that calcifications were stable, but short-interval follow-up was recommended. At that follow-up visit, radiologist thought that calcifications had increased in number since mammograms obtained 3.5 years earlier, although differences in technique were considered, and recommended biopsy. Pathology results were DCIS and calcifications were associated with carcinoma. Mediolateral oblique view. Photographic enlargement shows punctate calcifications (arrows) seen on mammograms obtained 3 years before study mammogram.

Figure 2.

Malignancy detected by human second reviewer in 52-year-old woman with ductal carcinoma in situ (DCIS) who presented with group of three or four punctate calcifications on screening mammograms. At time of screening, these calcifications had arguably been stable for 3 years. Therefore, human second reviewer's motivation to recommend additional views is unknown. At diagnostic visit, radiologist thought that calcifications were stable, but short-interval follow-up was recommended. At that follow-up visit, radiologist thought that calcifications had increased in number since mammograms obtained 3.5 years earlier, although differences in technique were considered, and recommended biopsy. Pathology results were DCIS and calcifications were associated with carcinoma. Mediolateral oblique view. Photographic enlargement shows calcifications (arrows) seen at screening; patient was called back by human second reviewer. Diagnostic workup concluded stability, but short-term follow-up was recommended.

Figure 3.

Malignancy detected by human second reviewer in 52-year-old woman with ductal carcinoma in situ (DCIS) who presented with group of three or four punctate calcifications on screening mammograms. At time of screening, these calcifications had arguably been stable for 3 years. Therefore, human second reviewer's motivation to recommend additional views is unknown. At diagnostic visit, radiologist thought that calcifications were stable, but short-interval follow-up was recommended. At that follow-up visit, radiologist thought that calcifications had increased in number since mammograms obtained 3.5 years earlier, although differences in technique were considered, and recommended biopsy. Pathology results were DCIS and calcifications were associated with carcinoma. Magnified (x1.8) mediolateral oblique view obtained 6 months after Figure 2 at time of biopsy that was recommended for same calcifications (arrows).

Figure 4.

Malignancy detected by human second reviewer: 73-year-old woman with ductal carcinoma who was called back by human second reviewer for possible architectural distortion versus summation shadows. Abnormality was suspected on only mediolateral oblique projection of screening mammographic images. Of note is that computer-aided detection (CAD) system had marked this same image, but mark had been dismissed by "CAD reviewer." At time of diagnostic evaluation, many additional views were obtained, and finding was considered to be superimposition of shadows. However, short-term follow-up was recommended in 6 months based only on radiologist's "gut" feeling, even though mammogram was considered to be negative for abnormal findings. At that follow-up, finding was now thought to be architectural distortion in two views but was best seen in craniocaudal projection. Whether this change represented progression in malignancy versus differences in projection is not known. Pathology showed ductal carcinoma in situ. Mediolateral oblique (Figure 4) and craniocaudal (Figure 5) mammograms. Photographic enlargements show area considered to be overlapping shadows (arrows) after diagnostic workup.

Figure 5.

Malignancy detected by human second reviewer: 73-year-old woman with ductal carcinoma who was called back by human second reviewer for possible architectural distortion versus summation shadows. Abnormality was suspected on only mediolateral oblique projection of screening mammographic images. Of note is that computer-aided detection (CAD) system had marked this same image, but mark had been dismissed by "CAD reviewer." At time of diagnostic evaluation, many additional views were obtained, and finding was considered to be superimposition of shadows. However, short-term follow-up was recommended in 6 months based only on radiologist's "gut" feeling, even though mammogram was considered to be negative for abnormal findings. At that follow-up, finding was now thought to be architectural distortion in two views but was best seen in craniocaudal projection. Whether this change represented progression in malignancy versus differences in projection is not known. Pathology showed ductal carcinoma in situ. Mediolateral oblique (Figure 4) and craniocaudal (Figure 5) mammograms. Photographic enlargements show area considered to be overlapping shadows (arrows) after diagnostic workup.

Figure 6.

Malignancy detected by human second reviewer: 73-year-old woman with ductal carcinoma who was called back by human second reviewer for possible architectural distortion versus summation shadows. Abnormality was suspected on only mediolateral oblique projection of screening mammographic images. Of note is that computer-aided detection (CAD) system had marked this same image, but mark had been dismissed by "CAD reviewer." At time of diagnostic evaluation, many additional views were obtained, and finding was considered to be superimposition of shadows. However, short-term follow-up was recommended in 6 months based only on radiologist's "gut" feeling, even though mammogram was considered to be negative for abnormal findings. At that follow-up, finding was now thought to be architectural distortion in two views but was best seen in craniocaudal projection. Whether this change represented progression in malignancy versus differences in projection is not known. Pathology showed ductal carcinoma in situ. Craniocaudal (Figure 6) and mediolateral oblique (Figure 7) mammograms (magnification, x1.8) 6 months later show architectural distortion (arrows) that prompted the radiologist to recommend surgical biopsy.

Figure 7.

Malignancy detected by human second reviewer: 73-year-old woman with ductal carcinoma who was called back by human second reviewer for possible architectural distortion versus summation shadows. Abnormality was suspected on only mediolateral oblique projection of screening mammographic images. Of note is that computer-aided detection (CAD) system had marked this same image, but mark had been dismissed by "CAD reviewer." At time of diagnostic evaluation, many additional views were obtained, and finding was considered to be superimposition of shadows. However, short-term follow-up was recommended in 6 months based only on radiologist's "gut" feeling, even though mammogram was considered to be negative for abnormal findings. At that follow-up, finding was now thought to be architectural distortion in two views but was best seen in craniocaudal projection. Whether this change represented progression in malignancy versus differences in projection is not known. Pathology showed ductal carcinoma in situ. Craniocaudal (Figure 6) and mediolateral oblique (Figure 7) mammograms (magnification, x1.8) 6 months later show architectural distortion (arrows) that prompted the radiologist to recommend surgical biopsy.

Interval Cancers Not Detected by Any of the Readers

There were three false-negative cases of malignancy arising within 12 months of the negative screening mammograms not noted by any of the readers. On retrospective review of the original screening mammograms, two of the three cases were considered negative by consensus. The third false-negative for all three readers was considered to be a missed case on retrospective review. This was in a 76-year-old woman who developed a malignancy in the scar of a previously excised benign mass from 4 years earlier (Figure 8, Figure 9 and Figure 10); the tumor was detected by palpation 9 months after the negative screening examination. It is noteworthy that the CAD system had marked this area, but the mark had been dismissed by the primary reader. The second human reader noted only postsurgical changes. In retrospective review, there were mammographic signs of malignancy in the scar characterized by increasing density at the biopsy site. A 2.1-cm invasive ductal carcinoma was surgically excised.

Figure 8.

False-negative case for all reviewers: 76-year-old woman with invasive ductal carcinoma. Craniocaudal mammogram obtained 4 years before study in which mass (arrow) was excised and was found to be benign (fibrocystic changes without atypia) at histology.

Figure 9.

False-negative case for all reviewers: 76-year-old woman with invasive ductal carcinoma. Screening mammogram, craniocaudal view, 2 years before study shows postsurgical changes.

Figure 10.

False-negative case for all reviewers: 76-year-old woman with invasive ductal carcinoma. Screening mammogram, craniocaudal view, at time of study in which increase in density at biopsy site was not detected by any of reviewers, although area was marked by computer-aided detection system.

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as:

processing....