Machine Learning Helps Identify Wasteful Medical Testing

By Marilynn Larkin

September 27, 2019

NEW YORK (Reuters Health) - Low-yield diagnostic testing is common, but machine learning could potentially identify and discourage wasteful testing that may be costly and harmful, researchers say.

"Clinicians and patients are routinely stuck making important medical decisions with only rough and often inaccurate impressions of risk," Dr. Jonathan Chen of Stanford University told Reuters Health by email. "The study illustrates how to systematically identify practices for change, and support clinical practice that is often drowning in tons of data, but little information."

"Many clear cases of wasteful repeated testing can be easily identified and should be immediately intercepted by practitioners today," he said. "We focused on the most common medical procedure, laboratory testing, with estimates that up to half of hospital lab tests are medically unnecessary."

"Implementation of the tools demonstrated in the study can guide clinicians towards which diagnostic tests are most -and least - likely to be useful, to provide the best quality and value of care for their patients," he added.

As reported online September 11 in JAMA Network Open, Dr. Chen and colleagues studied data on 116,637 inpatients treated at Stanford University Hospital from 2008 - 2017; 60, 929 treated at University of Michigan from 2015 - 2018; and 13,940 treated at the University of California, San Francisco from January through December 2018.

In 2014-2017 data sets from Stanford University Hospital patients (mean age, about 59; about half women), among the top 20 highest-volume tests, 792,397 were repeats of orders within 24 hours, including tests that are physiologically unlikely to yield new information that quickly - e.g., white blood cell differential, glycated hemoglobin, and serum albumin level.

The best-performing machine learning models predicted normal results with an area under the receiver operating characteristics (AUROC) of 0.90 or greater for 12 stand-alone laboratory tests, such as:

- sodium AUROC, 0.92; sensitivity, 98%; specificity, 35%; PPV, 66%; NPV, 93%;

- lactate dehydrogenase AUROC, 0.93; sensitivity, 96%; specificity, 65%; PPV, 71%; NPV, 95%;

- troponin I AUROC, 0.92; sensitivity, 88%; specificity, 79%; PPV, 67%; NPV, 93%.

Normal results were also predicted with an AUROC of 0.90 or more for 10 common laboratory test components, including:

- hemoglobin AUROC, 0.94; sensitivity, 99%; specificity, 17%; PPV, 90%; NPV, 81%;

- creatinine AUROC, 0.96; sensitivity, 93%; specificity, 83%; PPV, 79%; NPV, 94%;

- urea nitrogen AUROC, 0.95; sensitivity, 87%; specificity, 89%; PPV, 77%; NPV 94%.

Similar results were observed with the data from the other sites, Dr. Chen said. "Even when transporting prediction models from one site to another," he said, "significant predictive power is retained in most cases."

"Implementing machine learning models appear to be able to quantify the level of uncertainty and expected information gained from diagnostic tests explicitly, with the potential to encourage useful testing and discourage low-value testing that incurs direct costs and indirect harms," the authors conclude.

Dr. Chen said, "Health systems can identify, measure, and eliminate slam dunk cases of waste such as repeated tests for things that cannot credibly change that quickly - e.g., hemoglobin A1c, thyroid stimulating hormone, and albumin."

"They can also develop and execute data and analytic partnerships for other high-value targets, but which require more clinical patient context to identify," he added.

"Concerted effort is always needed to effect change in complex healthcare systems," he said. "The alignment of incentives to eliminate waste and free more resources to do more good for more patients should encourage health system administrators to work with clinician champions on such efforts, in partnership with data analytic teams to enable the technical infrastructure."

Dr. Maria E. Aguero-Rosenfeld, Director of Clinical Laboratories, NYU Langone Health and Professor, Departments of Medicine and Pathology, NYU School of Medicine in New York City, commented, "This is a very interesting paper, and I agree with the conclusions. There is no doubt that laboratory services are over-utilized and perhaps a lot is unnecessary testing."

"As indicated in this study, unnecessary testing adds cost, inefficiency, potential bad outcomes in patient care and patient dissatisfaction," she told Reuters Health by email.

"This is an area that NYU Langone Health has been involved via different avenues," said Dr. Aguero-Rosenfeld, who was not involved in the study. "We collaborate with our Value-Based Medicine team who has done analyses of utilization on inpatients in order to reduce repeat testing."

"We are going to use the analysis done on repeat CBCs and introduce a Best Practice Alert very shortly," she said. "We also established a Clinical Ancillary Services Committee that reports to the medical board. Its mission is to analyze utilization, review our testing menu, eliminate obsolete testing and approve addition of new tests."

"We also analyze utilization of send-out tests," she noted. "There is software available that could alert the ordering provider when a test might not be necessary based on prior results. These data-driven predictions might be very useful to limit the use of unnecessary testing."


JAMA Netw Open 2019.