Machine-Learning Methods Identify People at Risk of First-Episode Psychosis

By Will Boggs MD

April 09, 2020

NEW YORK (Reuters Health) - Machine-learning methods applied to electronic health records can identify individuals at risk of a first episode of psychosis (FEP), according to a new study.

"We were able to build a time-dynamic personalized risk-scoring algorithm (DETECT) that could detect individuals at risk for developing a first episode of psychosis one year prior to its occurrence," said Dr. Lars Lau Raket of Lundbeck, in Valby, Denmark, and Lund University, in Lund, Sweden.

"The detection method is passive in that it does not require any specific information to be collected to make its risk predictions, but instead relies on data from electronic health records (EHRs) that are collected as part of standard care," he told Reuters Health by email.

FEP is a key event that defines long-term outcomes in patients with schizophrenia, Dr. Raket and his colleagues write in The Lancet Digital Health. Once FEP occurs, there are only limited possibilities to improve long-term outcomes, they add.

The researchers developed the Dynamic Electronic Health Record Detection (DETECT) neural network and evaluated its accuracy for predicting FEP one year before its occurrence using data from IBM Explorys, which includes patient-level EHR data pooled from different U.S. healthcare systems. The FEP and control cohort each included more than 72,000 individuals.

DETECT predicted whether a FEP would occur one year before the index date with an accuracy of 78.7% in the development database, 77.4% in the validation dataset and 72.4% in the external validation subset.

Decision-curve analysis in the validation dataset suggested that detection based on the predictions made by DETECT were potentially associated with a positive net benefit for cost-benefit ratios below 1:3 for single-point risk assessment and below 1:16 for continuous-time risk assessment.

The most common events associated with the probability of FEP were somatic or unspecified healthcare encounters and encounters relating to psychiatric or brain health problems, including substance dependence or abuse, injury, assault, or self-harm.

"This tool still needs further development and validation before it can be implemented in clinical practice, but our analyses suggest that DETECT has an adequate level of accuracy to offer useful information for supporting clinical decision making," Dr. Raket said.

"Since DETECT is a low-cost approach that does not require collection of additional data to make risk predictions, it could be used in the context of sequential risk assessment to screen EHRs at large to detect individuals who could be referred to additional face-to-face assessment," he said.

"Analyses of big data will transform the healthcare system," Dr. Raket said. "By moving from studying hundreds of patients to thousands and even millions of patients, we can identify complex disease patterns and possibly come up with new ways to deal with the complexity of mental disorders."

Dr. Ioana Alina Cristea of the University of Pavia, in Italy, and Dr. Florian Naudet of the University of Rennes, in France, who wrote a linked editorial, told Reuters Health by email, "For psychiatry in general and FEP in particular, we remain unconvinced that at the present moment the benefits (of screening) outweigh the costs."

"For FEP specifically, there is as yet no tool with an adequate performance as a screening instrument for the general population," they noted. "The existent clinical-high-risk-for-psychosis approach has low positive predictive value. This problem is not overcome by DETECT, which, due to its 1:1 case-control design cannot give any reliable estimate of predictive value (positive or negative)."

"Even if DETECT were validated prospectively and at a population level, it is unclear whether it would be useful as a screening tool," they said. "Individuals identified by DETECT as having a risk for FEP will probably need to undergo clinical evaluation anyway, and it is also unclear what, if any, interventions should be offered to them. These are all aspects that will need to be assessed if the tool is validated at a population level."

"It is absolutely key to remain vigilant of the risk of overdiagnosis and overtreatment," Dr. Cristea and Dr. Naudet said. "If used at a large scale, a tool like DETECT, particularly if one takes an incidence of FEP of 26.6 per 100,000 person-years, as the authors assume in the report, might lead to a large number of false positives, overdiagnosis, and, conversely, unnecessary treatment. Of course, this might prove not to be the case in practice, but we do not have enough information at the moment to really estimate this risk."

"The history of screening in other areas of medicine should serve as a cautionary tale of how easy it is to be persuaded by the advantages promised by screening and overlook the sometimes considerable personal and societal harms," they concluded.

Lundbeck, which sells antipsychotic medication, funded the study and employed several of the authors.

SOURCE: and Lancet Digital Health, online March 26, 2020.