Facing, Not Fearing, Uncertainty in Diagnostic Testing

A Study in Diagnostic Subjectivity

Diagnosis, far from being an exact science, has an undeniable element of subjectivity. Elmore and colleagues^[1] illustrated this in a study comparing the accuracy and reproducibility of pathologists' diagnoses of melanocytic skin lesions. Slides from these lesions were interpreted by 187 pathologists from 10 different states, with each pathologist reviewing the same lesions on two different occasions to assess for variability in diagnoses.

These pathologists, who were experienced in the evaluation of melanocytic skin lesions, each reviewed 48 specimens. An average of 10 different diagnostic terms were applied to each specimen, although the suggested treatment was often the same. Interobserver concordance rates ranged from 25% for class II lesions to 77% for class V lesions.

In the second phase of the study, 99 pathologists reviewed the same set of 48 specimens 8 months later. The most mildly dysplastic cases (class I) were given the identical diagnosis 77% of the time by the same pathologist. The most severe lesions, including invasive melanoma (class V) received an identical diagnosis 83% of the time from the same pathologist. Class II, III, and IV lesions had intraobserver concordance rates of only 35%, 60% and 63% respectively.

This study acknowledges the challenge of classifying medical data based on subjective interpretations and the variability in pathology reports. These findings, say the investigators, support the use of a standardized classification system for the evaluation of melanocytic lesions.

Viewpoint

The process of diagnosis, on the surface, seems straightforward and, therefore, certain. A patient has a condition, which is evaluated with some type of testing. A diagnosis is made on the basis of these test results. If a test is positive, a diagnostic image reveals an abnormality, or a pathology report is definitive, the patient has the condition. But in reality, even with evidence-based medicine, a diagnosis is based on probabilities, not certainty. The study by Elmore and colleagues reminds us that, in real-world healthcare, the diagnostic process is inexact and uncertain.

Although used to support a diagnosis, pathology and radiology reports are really subjective opinions. They are influenced by objective criteria but also by the education and experience of the pathologist or radiologist. Many studies, like this one, show that concordance among pathologists and radiologists ranges from 50% to 80%.^[2,3,4] Laboratory tests carry uncertainty, as well. Even a test that has a sensitivity of 99.98% will produce 2 false positives out of every 10,000 positive results. With a test that has 90% sensitivity, 10 people out of 100 with positive test results will not have the condition that the test is trying to detect.

When the pretest probability of a condition is low, false-positive results will be common. When the pretest probability of a condition is high, false-negative results will be common. Screening tests are classic examples of the effects of low pretest probability conditions. A large population is screened with the expectation that only a few people will have the condition, but there are many false positives. For example, 96% of those who screen positive on low-dose CT scanning do not have lung cancer. Only about 4% of those with positive screening results actually have lung cancer.^[5]

Screening mammography, with a low pretest probability of disease, is similar. If a woman has a positive mammogram, what is the likelihood that she actually has breast cancer? Psychologist Gerd Gigerenzer asked 160 physicians this question: With a 1% prevalence of breast cancer in the population (pretest probability) and using a 90% sensitivity of screening mammography, what is the probability of breast cancer in a woman with a positive screening mammogram? Sixty percent responded that a positive mammogram meant a ≥80% probability of having breast cancer. The actual probability of breast cancer is only 9.2%.^[6]

Uncertainty should not result in fear but in realism. Simply recognizing that there are uncertainties in healthcare can lead us to think differently. Uncertainty leads us to incorporate clinical judgment and experience into the diagnostic process, recognizing that diagnostic tests are tools but should not overrule clinical judgment. Using available data along with information about probabilities and clinical intuition, the clinician can engage patients to make decisions about their care.

Comments

Commenting is limited to medical professionals. To comment please Log-in.

Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.