Google Glass App Helps Kids With Autism Read Facial Expressions

Megan Brooks

August 10, 2018

Children with autism can learn to read facial expressions by using a smartphone app that runs on Google Glass. The app provides real-time cues about other people's facial expressions, according to researchers who conducted a small pilot study.

After 1 to 3 months of regular use, parents reported that children with autism made more eye contact and related better to others after using Superpower Glass, which was developed by a team from Stanford University School of Medicine in Palo Alto, California.

Google Glass app helps children with autism read facial expressions. Steve Fisch/Stanford School of Medicine

"The results are very encouraging," Dennis Wall, PhD, associate professor of pediatrics and biomedical data science at Stanford, noted in an interview with Medscape Medical News.

The study was published online August 2 in NPJ Digital Medicine.

Eight Core Facial Features

The device consists of "smart" glasses equipped with a camera to record the child's field of view, as well as a small screen and a speaker to give the child visual and audio information. As the child interacts with others, the app identifies and names their emotions through the Google Glass speaker or screen. It's designed for use in the child's natural environment during social interactions with friends and family members.

The Stanford team used machine learning to enable their app to recognize eight core facial expressions: happiness, sadness, anger, disgust, surprise, fear, neutral expression, and contempt.

Users can choose from among three different ways to use the face-recognition program. In "free play," children wear Google Glass while interacting or playing with their families. The software provides the wearer with a visual or auditory cue each time it recognizes an emotion on the face of someone in the field of view.

There are also two game modes. In "guess my emotion," a parent acts out a facial expression that corresponds to one of the app's eight core facial expressions, and the child tries to identify the emotion. The game helps families and researchers track children's improvement at identifying emotions. In "capture the smile," children give another person clues about the emotion they want to elicit, until the other person acts it out. This helps the researchers gauge the children's understanding of different emotions.

In the pilot study, 14 families used Superpower Glass therapy at home for an average of 10 weeks. Each family had a child between the ages of 3 and 17 years who had received a clinically confirmed diagnosis of autism. The families used the device for at least three 20-minute sessions per week. At baseline and study completion, parents completed the Social Responsiveness Scale 2 (SRS-2), a peer-reviewed standardized measure to assess the severity of autism spectrum disorder (ASD) on the basis of social interaction.

Significant Gains in Social Skills

The mean total SRS-2 score was 80.07 at baseline (a higher score indicates higher severity of ASD). It improved to 72.93 by the end of the study period. Children's total SRS-2 scores decreased an average of 7.38 points during the course of the study.

For six of the 14 children, the decrease in score was large enough to warrant reclassifying the severity of ASD by one step: for four patients, ASD was reclassified from "severe" to "moderate"; one patient's ASD was reclassified from "moderate" to "mild"; and one patient's ASD was reclassified from "mild" to "normal." During semistructured interviews, 12 of the 14 families reported increased eye contact and greater social acuity, the authors report.

"The SRS showed significant gains pre- and post intervention, but it's important to note that this is only a feasibility study and not an efficacy study, because there was no control arm," Wall told Medscape Medical News.

The researchers have just completed a randomized controlled trial involving 74 children in which they achieved "quite similar results," Wall said. He said the results of this study will be submitted for publication in the next few weeks.

"We now have a fairly good body of evidence to support moving it into a commercial setting and think about regulatory approval and medical reimbursement," said Wall. Stanford University has filed a patent application for the technology.

Wall said this type of mobile therapy for ASD could help fill a gap in autism care. Because of a shortage of trained therapists, children may wait more than a year after being diagnosed with autism to begin receiving treatment. "Ideally, this is something that could be used at home while the children are wait-listed," Wall said.

Commenting on the findings for Medscape Medical News, Victor Fornari, MD, director of child and adolescent psychiatry, Zucker Hillside Hospital, Glen Oaks, New York, and Cohen Children's Medical Center, New Hyde Park, New York, noted that children with ASD "struggle with making eye contact, and the currently accepted best practices for intervention have limitations. New strategies to enhance social skills in youth with ASD are needed."

In this feasibility pilot study, the device was well tolerated, and families reported improved eye contact with their children. "Although there were limitations to the study design, the report suggests that further investigation of the benefits of wearable technology to enhance social skills in children with ASD is promising and warranted," said Fornari.

The research was supported in part by the National Institutes of Health, the Hartwell Foundation, the Bill and Melinda Gates Foundation, the Coulter Foundation, the Lucile Packard Foundation and by program grants from Stanford ' s Precision Health and Integrated Diagnostics Center, the Beckman Center, Bio-X Center, the Predictives and Diagnostics Accelerator Program, the Child Health Research Institute, and Human-Centered AI. In-kind material grants included a gift from Google (35 units of Google Glass version 1) and Amazon Web Services founder support. The authors have disclosed no relevant financial relationships.

NPJ Digital Med. Published online August 2, 2018. Full text

For more Medscape Psychiatry news, join us on Facebook and Twitter.


Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.