Brief Home Videos May Accurately Diagnose Autism in Kids

Batya Swift Yasgur, MA, LSW

December 04, 2018

Brief home videos may be effective in diagnosing autism spectrum disorder (ASD) in children, new research suggests.

Investigators tested eight machine learning models to identify which features of a child's behavior should be used to evaluate for potential ASD from brief home videos.

The model with the most superior performance identified ASD with a greater than 90% overall accuracy, including correctly labeling almost all children with ASD as having ASD and over three-quarters of children without ASD as not having it.

"Our main motivation for conducting the study was the long and expanding waiting list for access to diagnostic evaluations for children with potential ASD diagnoses," senior author Dennis Wall, PhD, associate professor of pediatrics and of biomedical data science at Stanford University School of Medicine in California told Medscape Medical News.

"Using home videos diagnostically can potentially streamline the diagnostic process, making it more efficient and enabling earlier diagnosis and hopefully resulting in earlier intervention," he said.

The study was published online November 27 in PLOS Medicine .

Diagnosis Cumbersome

ASD is diagnosed through behavioral exams using instruments such as the Autism Diagnostic Observation Schedule (ADOS) and the Autism Diagnostic Interview-Revised (ADI-R), the authors write.

The current diagnostic standard of care is "long and cumbersome, must be done at clinical facilities by certified clinical teams consisting of two, three, or four individuals — all amounting to a lengthy and costly process, 10 hours on average, sometimes spanning two days at a clinical facility that can be geographically hard to get to," Wall said.

This leads to a backup in the diagnostic process, so that many children are not diagnosed until they reach 5 years old.

"A child diagnosed at 5 could have possibly been diagnosed as early as 2, and the delay is in large part due to methods we use for diagnostic evaluations," Wall continued.

Children with ASD "respond extremely well to early intervention, and [age] 5 is close to the end of brain development, during which therapy could have had its maximal impact," he noted.

With an eye toward simplifying and streamlining the diagnosis, "we've applied machine learning to data to ask a simple question: can we reduce the complexity of the diagnostic process without any loss of accuracy?"

He recounted that his group has been using machine learning approaches to identify "minimal sets of behaviors that align with clinical diagnoses of autism."

Earlier studies identified eight classifiers.

Of these classifiers, two were validated in previous studies, supporting the hypothesis "that models using minimal feature sets are reliable and accurate for autism detection."

Window of Opportunity

The current study, built on this prior work, tests the hypothesis that the features represented in the minimal viable classifiers "can be labeled quickly, accurately, and reliably from short home videos by video raters with no official training in autism diagnosis or child development," the authors state.

The researchers used crowdsourcing and "real-time video analysis" to run feature labeling and evaluate the accuracy of all eight machine learning models trained to detect autism in two independent home video repositories.

Videos were between 1 and 5 minutes long, displayed the child's face and hands, and showed opportunities for social engagement as well as use of objects such as toys, crayons, or utensils.

The mobile web portal assessed 30 behavioral features in a yes/no format (eg, whether children were engaging in eye contact, social smiling, playing appropriately with toys) for video raters to assess.

The study used 116 short home videos of children with ASD (mean age [standard deviation (SD)], 4 years 10 months [2 years 3 months]) and 46 videos of nonautistic, typically developing children (mean age [SD], 2 years 11 months [1 year 2 months]).

At first, nine raters who were blind to the diagnosis rated all 30 questions from the eight models.

After they had scored 50 of the videos, the investigators determined that three raters were the minimum number required to generate a reliable score, so each remaining video was then scored by three raters.

The rating process took a median of 4 minutes to complete.

Using the data for each video, the researchers tested several models and found that a sparse five-feature logistical regression classifier (LR5) yielded the highest accuracy (area under the curve, 92%; 95% confidence interval, 88% - 97%) across all ages tested.

Machine Learning Models

The researchers then used a prospectively collected independent validation set of 66 videos (33 ASD and 33 non-ASD) and three independent rater measurements to validate the outcome.

LR5 again yielded the highest accuracy, with a sensitivity of 87.8% and a specificity of 72.7%.

Of the 66 videos, 13 were misclassified, including four false negatives. The researchers hypothesized that these videos might contain "challenging displays of autism symptoms."

After examining the probabilities generated by the LR5 model for the 13 misclassified videos and adjusting the probability threshold, 26 videos fell into the inconclusive category.

Once these were excluded from the accuracy analysis, the sensitivity and specificity increased to 91.3% and 88.2%, respectively.

"Short home videos can provide sufficient information to run machine learning classifiers trained to detect children with autism from those with either typical or atypical development," the authors note.

"Features needed by machine learning models designed to detect autism can be identified and measured in home videos on mobile devices by nonexperts in timeframes close to the total video length and under 6 minutes," they continue.

"Readings are done online in mobile portals, so the production of videos can scale to meet the demand with high efficiency and enable decision support,” Wall said.

"That enables clinicians to make clinically valid decisions and render diagnoses not in days or hours but in minutes, and do so in a way that drives down the average diagnosis age from where it is today to 2 years old. This will enable us to get right in there, in the window of brain development where it matters the most," he continued.

Room for Progress

Commenting on the study for Medscape Medical News, Jorge José, PhD, James H. Rudy Distinguished Professor of Physics, Indiana University, Bloomington, member of the Stark Neuroscience Research Institute, and faculty at Indiana University Medical School, who was not involved with the study, called it "very interesting" and "a very difficult thing to do."

Too many children with ASD "are not seen early enough, so one way of having a preliminary assessment of neurodevelopment would be looking at videos that parents can produce on YouTube, for example, by using common standard machine learning techniques to try to find out if children fulfill some of the characteristics associated with ASD," José said.

He cautioned that there is "still a lot of room for progress and research, given the fact that characteristics that are used by psychiatrists and clinicians in the ADOS and ADI-R are qualitative in nature and observational."

There is "a certain need for more quantitative ways of detecting and diagnosing, such as the ones our group has been studying — for example, arm movements — which don't depend on visual observation," José emphasized.

Wall added that future research will build on the current findings and focus on more structured videos and game-based technologies.

The work was supported in part by funds from National Institutes of Health (NIH), the Hartwell Foundation, Bill and Melinda Gates Foundation, Coulter Foundation, Lucile Packard Foundation, and program grants from Stanford University's Human Centered Artificial Intelligence Program, Precision Health and Integrated Diagnostics Center (PHIND), Beckman Center, Bio-X Center, Predictives and Diagnostics Accelerator, and the Child Health Research Institute. The research also received philanthropic support from David Orr, Imma Calvo, Bobby Dekesyer, and Peter Sullivan. Wall is the scientific founder of Cognoa, a company focused on digital pediatric healthcare; the approach and findings presented in this paper are independent of/not related to Cognoa. The other coauthors and José have disclosed no relevant financial relationships.

PLOS Med. Published online November 27, 2018. Full text

For more Medscape psychiatry news, join us on Facebook and Twitter


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.