Critics Question Study's 47% Observer Error Rate for Breast MRI

James Brice


June 29, 2012

Questionable Observer Error Rate for Breast MRI

French Study Investigates Reasons for Breast MRI Reading Errors

Editors of Radiology and the authors of a study on false-negative breast MRIs[1] have drawn criticism from prominent radiologists, who say that the article in the April 2012 issue of the widely read journal inappropriately encourages physicians to question the diagnostic accuracy of the modality.

Radiologist Emmanuelle Bouic Pagès, MD, and colleagues at CHU Lapeyronie, Montpellier, France, reported potential observer error in 47% of breast MRI studies performed from January 2005 to December 2010. CHU Lapeyronie is a large hospital near the Mediterranean coast in south-central France.

The intensive analysis of the hospital's experience with breast MRI is based on a comparison between a prior negative study and subsequent diagnostic MRI that revealed 60 cases of cancer in 58 women (average age, 48 years; range, 40-78 years at diagnosis).

On the basis of the findings, radiologists at CHU Lapeyronie identified various pitfalls of reading diagnostic MRI that in their opinion led to 28 overlooked, misinterpreted, or mismanaged cases of breast cancer.

But Christiana Kuhl, MD, University Medical Center, Aachen, Germany, and Elizabeth Morris at Memorial Sloan-Kettering Cancer Center, New York, New York, had few good things to say about the study. They even objected to the title, "Undiagnosed Breast Cancer at MR Imaging: Analysis of Causes."

"The title is misleading," Dr. Kuhl told Medscape. "It sounds like MRI misses cancer. Instead, it is a retrospective romp to see whether any dot of enhancement on MRI may have been the cancer."

Dr. Morris concurred. "This title is just terrible because it isn't really reflective of what the paper is about," she said in an interview with Medscape. "From that title, its sound like MRI is not picking up breast cancer, and that is not the point."

In contrast, Herbert Y. Kressel, MD, editor of Radiology, said that the title tells readers exactly what the study is about.

"Machines don't make diagnoses; people do," he explained to Medscape."This is a study about errors made by people using the results of MRI exams."

Dr. Pagès did not respond to e-mail requests to answer questions about the study.

If Drs. Kuhl and Morris seem particularly sensitive about Pagès and colleagues' negative review, it could be because both were involved with the 20-year uphill climb to establish clinical acceptance of MRI as a diagnostic instrument for breast imaging. Over time, its users have had to beat down criticisms about the relatively high costs of breast MRI compared with x-ray mammography and ultrasonography and its relatively high false-positive rates, despite its higher sensitivity, for breast cancer compared with other imaging techniques.

Breast MRI has gained a place in clinical practice, however, first for presurgical use to identify multifocal cancers and those in the contralateral breast, and then for screening high-risk women with dense breasts or a genetic propensity for breast cancer. Researchers are optimistic about the broad future adoption of dynamic contrast MRI for monitoring the response of locally advanced breast cancer to chemotherapy before surgery.

Previous studies have addressed why one third to one half of biopsy findings after a positive MRI are benign. The French study is one of a few controlled scientific inquires to examine the source of false-negative results with the modality. It involved a prospective reexamination of negative MRI studies and a retrospective unblinded comparison of those negative examinations and subsequent positive MRI scans to identify the cause of false-negative results. The 2 examinations were done a mean of 18 months apart.

The evaluation identified false-negative findings for 37 enhancing masses, 16 cases of non-masslike enhancements, and 7 foci. Forty-eight tumors were infiltrative; 12 were ductal carcinoma in situ.

The interpretation was considered adequate in 32 of 60 cancer cases (53% accuracy) from the prior MRI studies. No enhancement appeared in 11 cases at the site where an enhancing lesion subsequently appeared. Bilateral and multiple foci without any predominant or patchy enhancement were represented in 15 cases. In 6 cases, an isolated focus was accurately graded as Breast Imaging Reporting and Data System (BI-RADS) 3.

For 28 lesions, however, an abnormality was observed retrospectively that changed the BI-RADS classification.

Misinterpretation produced most of the false-positive results, according to the study. For 15 cases, the lesions were correctly identified but were incorrectly interpreted as a benign lesion or benign enhancing tissue. The mean size of the lesions was 9.5 mm (range, 4-17 mm). In both the retrospective and the prospective analysis, lesions appeared as masses in 8 cases, as a non-masslike enhancement in 5 cases, and as a focus in 2 cases.

"We can learn from that," Dr. Morris said. "The lesion is there. You can't be cavalier and just assume that it is a benign finding without first performing a complete analysis and possible biopsy."

But she believes that the authors were unrealistic in categorizing some initial scans as misinterpretations. She noted than any one of several "dots" appearing in part (a) of the Figure, which was published in the study, could have been read as suspicious. The 3 images in the Figure may have provided an example of a stable lesion that evolved into cancer over time, but accurately characterizing it as suspicious on the initial or follow-up scan (Figure) would have been impossible because the lesion did not reveal any anatomic signs of malignancy, she said.

Figure. Breast image from the study by Pagès and colleagues. (a) The investigators rated the lesion identified by the arrow as potentially misinterpreted. (b) The lesion remained stable after 6 months, as seen on this T1-weighted gradient echo sequence subtracted image; (c) however, a 20-mm speculated mass corresponding with grade 2 invasive ductal carcinoma appeared on MRI 2 years later. (Republished with permission of the Radiological Society of North America)

"You would never biopsy that lesion," Dr. Morris said. "To say that you missed it is the wrong interpretation."

Seven cases of false-negative findings were considered to have been mismanaged owing to biopsy-related issues. Four instances of poor lesion targeting or bleeding that interfered with tissue collection stemmed from ultrasonography-guided biopsy. One case of poor targeting arose from MRI guidance, and in another case, ultrasonography and MRI targeting were used. Finally, an instance of denied biopsy was also considered mismanagement.

Six cancerous lesions were not recognized from reading of the prior examination because of their small size, high background enhancement, or a preaxillary seat. Their mean size was 6.5 mm (range, 3-13 mm).

Dr. Pagès and colleagues attributed some reading errors to differences in the presentation of malignant masses on MRI and ultrasonography. They pointed to a 2002 study from Memorial Sloan-Kettering Cancer Center that reported carcinoma in 17% of smooth margin masses at MRI, although smooth margins are primary indicators of benign abnormalities on ultrasonography and x-ray mammography.[2] Pagès and colleagues correlated smooth margins on MRI images with mammographic or ultrasonographic findings before concluding that the lesion was benign.

The researchers appreciated the known relationship between the presence of enhancing internal septations and the high likelihood of fibroadenoma.

The CHU Lapeyronie group also noted that breast tumors can be stable yet still be malignant. Previously published work by the group showed that the growth rate of breast cancer can be highly variable. In the current study, they found that periods of stability may last as long as 6 months. They cite this as the main reason for errors in 3 of the 15 misinterpreted cases.

In terms of mismanaged cases, the researchers conceded that they mainly arise because of inadequate correlations between enhancement on MRI and the image used for ultrasonography-guided biopsies. The authors stressed the importance of the radiologist's attention when determining that findings on targeted ultrasonography correspond with lesions on MRI, citing a 2003 lung cancer study in which ultrasonography-guided biopsy did not correspond with the targeted MRI lesion in 12.5% of cases.[3]

Yet, Dr. Kuhl saw nothing in the CHU Lapeyronie study that could not already be found in the BI-RADS lexicon. This includes general professional understanding that round masses can be cancer and that nonenhancing septation may be seen as cancer.

Dr. Morris was surprised that the false-negative rate at CHU Lapeyronie reflected the effect of missed cancer from poorly targeted biopsies when its radiologists did not comply with a fundamental principle of biopsies for suspicious breast masses.

"If you see it on MRI, biopsy it," Dr. Morris said. "You shouldn't have a situation where you mismanage lesions."

Still, all biopsies involve some misses, she noted. The miss rate for MRI-guided biopsies at Memorial Sloan-Kettering is 4%, which Dr. Morris attributed to the small size of the lesions and to targeting difficulties.

Drs. Kuhl and Morris also worried that an overemphasis on reducing false-negative findings could lead to more false-positive results on breast MRI. "That is something we definitely don't want to do," Dr. Morris said.

But Radiology editor Dr. Kressel stressed that there is no doubt that 60 cases of cancer in the 58 pairs of studies evaluated by Dr. Pagès and colleagues were not diagnosed. He considers the study to be relevant, albeit unusual, because it reveals the types of problems that radiologists encounter in routine clinical practice.

"This is study is a snapshot as to the cause," he said.

None of the study's authors or independent sources had potential conflicts to report.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: