Some Lab-Developed Tests May Lead to Bad Outcomes, Says FDA

December 03, 2015

A category of lightly regulated and sometimes flawed diagnostic tests needs more federal oversight, lest they continue to mislead patients into undergoing needless surgeries, including mastectomies, or foregoing needed care, officials from the US Food and Drug Administration (FDA) told Congress last month.

The FDA's call for action comes as the agency is cracking down on such tests, as the companies 23andMe and Theranos know well. Ironically, many of the tests deemed imprecise are examples of the precision medicine movement, which aims to personalize treatments based on an individual's genetic and molecular profile.

The FDA wants stricter standards for tests that are designed, manufactured, and used within a single laboratory, in contrast to off-the-shelf tests that laboratories buy from other companies. The latter kind usually require some sort of FDA approval based on proof of safety and effectiveness before they can go on the market. Commercially sold tests classified as high risk must pass a rigorous "premarket review." The FDA can greenlight moderate-risk tests if a so-called 510(k) review determines they are basically equivalent to a product already on the market.

With laboratory-developed tests (LDTs), however, the FDA has exercised "enforcement discretion" when it comes to this vetting process, at least up until now. LDTs traditionally were considered low risk because they were relatively simple and confined to a local laboratory. Many focused on rare diseases.

However, complex LDTs for common and serious conditions such as cancer and coronary heart disease raise the stakes for patients, especially when the tests are nationally available. Accordingly, the FDA has proposed subjecting high-risk LDTs to the same premarket review that high-risk, commercially sold tests must pass.

"Doctors and patients don't care about who makes their tests," Jeffrey Shuren, MD, director of the FDA's Center for Devices and Radiological Health, told the health subcommittee of the House Energy and Commerce Committee on November 17. "They do care if their tests are accurate, reliable, and clinically valid."

Most of Problematic LDTs Cited by FDA Involve Cancer

To add weight to its proposal for stricter oversight, the FDA gave the committee a new report highlighting 20 case studies of problematic LDTs. Their weaknesses include high rates of false positives, false negatives, or both; testing for factors with no clear relevance to a disease; links to scientifically unsound treatments; and lack of analytic or clinical validation.

Eleven of the 20 studies involve cancer, either in terms of identifying people at an elevated risk for it, detecting it, or guiding treatment. OvaSure, no longer on the market, screened for ovarian cancer using a blood test for four biomarkers, for example.

Table. How Laboratory-Developed Tests Can Mislead

Test Problem Clinical Consequence
High rate of false positives
OvaSure ovarian cancer screening test No validation that the test predicts or detects ovarian cancer; inflated claims about positive predictive value Women with false positives may undergo unnecessary surgery to remove healthy ovaries
High rate of false negatives
Human papillomavirus test using SurePath collection medium Unknown sensitivity Patients with false negatives may receive improper care, precancerous cells may turn cancerous, and patients may need more extreme interventions
High rate of false positives and false negatives
Noninvasive prenatal testing (aka cell-free DNA testing) Lack of clinical validation that the test detects and predicts fetal abnormalities at an appropriate rate; many false positives in the general population Women with false positives may abort a normal pregnancy; women with false negatives may deliver a child with unexpected genetic syndrome
Factors with no clear disease relevance
KIF6 genotyping test to predict heart disease risk and statin therapy response Biomarker not adequately shown to predict coronary heart disease or statin response; test incorrectly validated; unproven product claims Overtreatment or undertreatment with statins
Links to treatment based on disproven scientific concepts
CARE Clinics autism biomarkers test No evidence that "causes" identified by test correlate to autism spectrum disorder; no evidence that recommended treatments improve outcomes Children undergo inappropriate and harmful treatments based on test results
Undermining of drug approval or drug treatment selection
Duke University chemotherapy assessment test Errors in data management and analysis; lack of clinical validation that test predicts response Patients enrolled in trials took unproven, potentially inadequate treatments
Lack of validation
OncoVue genetic breast cancer risk test Specificity not assessed; lack of validation of test performance in clinical use Patients with elevated scores may undergo unnecessary mastectomy or tamoxifen prophylaxis, patients with low scores may forego recommended screenings

Source: "The Public Health Evidence for FDA Oversight of Laboratory Developed Tests: 20 Case Studies." FDA.

OvaSure was a poor screening test, the FDA said in its report. There was no validation that it actually predicted or detected ovarian cancer, and the test produced a lot of false positives. Its creators had originally claimed a positive predictive value of 99.3%, meaning that virtually all positive results appeared to represent actual cases of ovarian cancer. However, that calculation did not reflect the prevalence of ovarian cancer in the general population, which lowered the positive predictive value to only 6.5%, meaning that most positive results were false. Women with these false positives "could undergo unnecessary surgery to remove healthy ovaries, if subsequent workup did not rule out the disease," the agency said.

Another LDT for cancer singled out by the FDA is OncoVue, which predicts inherited breast cancer risk. Although the test was hailed for its higher-than-usual sensitivity, its ability to exclude the disease when absent, called a test's specificity, has never been tested. That leaves open the question of false negatives. The FDA said that a woman receiving a high risk score in error could go so far as to have her breasts removed to prevent cancer.

Other conditions covered by the LDTs in the FDA report included heart disease, autism, pertussis, and fibromyalgia.

23andMe a Poster Child for LDT Regulation

Although the FDA is proposing to subject high-risk LDTs to premarket review of their safety and effectiveness, the agency already has started to separate the sheep from the goats.

Last year, for example, the agency ordered 23andMe to pull its health tests from the market because the company's saliva collection kit and "personal genome service" lacked FDA approval. The casualties were risk tests for diseases ranging from Alzheimer's to breast cancer, tests for responses to drugs such as metformin and statins, and carrier status tests. 23andMe sells its tests directly to consumers.

In February, the FDA approved the company's carrier status test for an autosomal recessive condition called Bloom Syndrome. The decision came down not after a premarket or 510(k) review, but after a "de novo" regulatory process reserved for novel products with low to moderate risk. At that time, the agency also said it intended to exempt all carrier status tests for autosomal recessive conditions from any kind of premarket approval, given their relatively low risk. That announcement gave 23andMe an opening to reintroduce all of its carrier status tests for autosomal recessive conditions in October. However, its higher-risk tests remain in limbo.

A variant of that story has played out with Theranos, which offers low-cost finger-prick blood tests using mini collection vials. These, too, are LDTs, and one for herpes simplex has been cleared by the FDA. However, the FDA recently notified the company that its patented Nanotainer vial is an unapproved medical device. Until it receives that green light, the company is substituting venous collection tubes for the Nanotainers in all its blood tests except the one for herpes simplex.

"Completely Out of Context"

The path the FDA is charting for LDTs has struck some quarters of the healthcare world as potentially reducing diagnostic innovation and access to new tests. A coalition that includes the American Medical Association, the National Independent Laboratory Association, and the American College of Medical Genetics and Genomics (ACMG) sent a letter to House and Senate leaders in October urging them to improve the accuracy and clinical value of LDTs through an existing regulatory system, referred to as the Clinical Laboratory Improvement Amendments (CLIA). Modernizing CLIA for the precision medicine era, they said, is preferable to the FDA "imposing an entirely new regulatory regime" on the healthcare industry.

ACMG Executive Director Michael Watson, PhD, told Medscape Medical News that not all the LDTs in the FDA's 20 case studies are as faulty as the agency makes them out to be. Some of the case studies on screening tests are "completely out of context," said Dr Watson.

"You may complain that [an LDT] doesn't find everyone who has the disease," he said. However, screening tests are not designed to be the final word.

"It's just the first step in the process," said Dr Watson. "People get screened, and then they get tested to see if what was screened is actually there or not."

With other tests cited by the FDA, the problems are not so much a matter of test accuracy as test interpretation, which has to do with the practice of medicine, he said. "That's outside the FDA's jurisdiction."

The ACMG has no problem with the FDA overseeing high-risk LDTs, Dr Watson said. "Where we differ is which tests are high risk."

In the past, the FDA has put the high-risk label on any test that predicts disease in an asymptomatic individual, he said. "We don't think it's always high risk because there is often other information that informs the test result."

LDTs that detect harmful BRCA gene mutations are one example, said Dr Watson. Clinicians never view these test results in isolation, but instead factor in imaging studies and family history, among other things.

The cautionary letter from ACMG and the other health organizations buttressed the regulation-lite views of the Republicans in charge of the health subcommittee of the House Energy and Commerce Committee. At last month's hearing, subcommittee chair Rep. Joe Pitts (R-PA) seemed to borrow from the coalition letter when he said that he does not believe "in imposing a new regulatory reality on an increasingly important component of our healthcare industry."

Pitts and other members of his subcommittee seek to streamline — and some would say weaken — the FDA approval process for new drugs and medical devices through the 21st Century Cures Act. That bill, born in the House Energy and Commerce Committee, passed the House in July with strong bipartisan support. The Senate is crafting its version of the legislation.

In the spirit of the 21st Century Cures Act, another Republican member of the health subcommittee also warned against the FDA's plan to regulate LDTs.

"We're talking about proposals that may not just stifle, but eliminate medical innovation," said Rep. Michael Burgess, MD (R-TX). "And we're also opening the door to the federal regulation of the practice of medicine, not the needles and [intravenous] solutions, but the actual diagnostic thought processes that go into the practice of medicine."


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.
Post as: