Brain Power Has Created A Novel Google Glass Autism application

Brain Power Has Created A Novel Google Glass Autism application

The month of April is Autism Month. Autism affects 3.5 million families in the United States alone, with each person uniquely on the spectrum. These people commonly experience symptoms related to attention-deficit/hyperactivity disorder (ADHD), including hyperactivity, inattention, and impulsivity, but more well known is their struggle to decode emotions, to manage transitions and behaviors, eye contact, casual conversation, and so on.

Brain Power was founded to address autism through a heads-up wearable computer, in the form of Google Glass, that delivers gamified apps used to motivate and reward users for social and cognitive learning. The use of augmented reality has been coupled with artificial-intelligence systems to aid people with autism-related challenges in various ways that extend beyond cartoonish games.

We had an opportunity to speak with Dr. Ned Sahin, founder and CEO, and to Dr. Arshya Vahabzadeh, Brain Power’s Chief Medical Officer, to find out about the technology and how it’s helping those with autism.

Alice : Thanks for joining me today. Can you please tell me a bit about yourself, and how you came up with this idea? And why did you pick Google Glass?

Dr. Ned Sahin, Founder & CEO: I am the founder and the inventor and they came up with the idea in 2013 just when Glass was new and everyone was talking about it. But it wasn’t to be a Glass developer, it was to find a way of applying good science and new technology to helping humankind. I wanted to find a way that could impact people in their daily lives. My personal history through Ph.D. at Harvard and postdoc and so forth was one of a typical and successful scientific career. I felt nonetheless that there needed to be something that makes contact with people who go through these challenges in their lives. There are people who just suffer and it’s not regular life challenges – it’s that their inner world is not being recognized by the outside world. We have so many autistic team members here at Brain Power and just over lunch yesterday we were talking about how everyone feels locked in and misunderstood by the world and how technology can unlock the inner capabilities. And that was my day dream then, and it is my dream now, to see the reality of having autistic people working at Brain Power. It has been rewarding to see the hundreds of people who use our technology, such as parents, feel more connected with their own children, and to hear teachers say they can now teach more rather than “police” in a behavioral way in the classroom. So it’s super exciting and it’s all based on how to make the biggest impact for people.

To answer your question about Glass, it keeps people actually in their world, and not in an escapist pod blocked away from the world. Whether Google made the perfect thing with Glass and whether there were some issues there are really not for me to talk about. It’s a light device that’s rugged and brings the computer into the longer sight and brings the person back into the world. I mean how many cartoons and commercials decry that everyone is stuck in their phones and tablets and it’s partially true but with a heads-up device you are in the world and hands free and engaged with family and friends and so forth. That’s the big difference.

It’s really having looked around and seeing that this is a huge need in this space. The healthcare system is not addressing this, and the family needs are not being addressed, and an educational system cannot fill this gap either.

 Here is a simple vignette. The child puts on the device and looks back at mom or the teacher, and now is seeing the mom’s face outlined with a circle or covered at first with a fun cartoon. Whether it’s a character like Olaf (not copyrighted of course), the cartoon draws the child’s attention to it and then gives a bit of a reward. But then the child needs to spend time looking and gets rewarded with gold stars and leveling up by first of all just maintaining that gaze toward the person, and then eventually by choosing an emotional expression. For instance, it might have a surprised face and a happy face in cartoon forms floating on either side – those are the AR elements, and the AI has determined what the face is actually showing at that time, and the child tilts their head or uses a verbal command to determine which one it was and gets rewarded accordingly. This is picked up by the accelerometer or microphones and was a simple enough gesture, but we elaborate the gamified elements to keep it motivational.

We have tested in on so many people with autism, both young and old. It is motivational, not a burden – it’s safe, it’s exciting, and has clear improvements. That’s some aspects of it. You asked about the artificial intelligence, so part of that is to detect the classified facial expression and presence of faces. Additionally, we use body signs of the wearer in terms of things like heart rate and physiological signs to determine what state that person is in. Is this for instance boring him? Or stressing him more? And then we customize based on that.

Medgadget: So do you have wearables you use too? E.g., for heart rate? I’d also like to hear more about your Emotion Charades app.

Dr. Sahin: Right now we do a lot with Google Glass, and so it’s mostly through the Glass and our apps.

In the Emotion Charades app, the child sees an emoji on either side of the face of a happy versus angry face. The child then chooses an emoji by tilting their head or saying the word and gets rewarded if the answer is correct. 

Dr. Arshya Vahabzadeh, Chief Medical Officer: We used classical clinical scales – the ABC-H behavior checklist, which is one of the main checklists used in treatment outcomes studies associated with autism. We use that ABC-H scale to identify all the baseline performances of these individuals into two different categories of a high ADHD-related symptom group and low ADHD-related symptom group. From there, we give users a prolonged intervention using the glasses where they get to go through a series of modules that focus on things like facial and emotional tension and recognizing facial emotions. Based on that, there’s a subsequent ABC-H scale used at 24 hours and 48 hours. We identify a clinical response as being a 25% improvement in the numerical value, which is something that’s been previously reported. We have found that not only did the ABC-H scores decrease, but they just decreased to an amount that would be identified as a clinical response.

Medgadget: Is the social and emotional coaching done in real-time? And was it difficult to quantify the findings given that things such as social communication and behavior in people are more subjective rather than objective?

Dr. Vahabzadeh: No, the socioemotional coaching is part of the software. And no, instead of just using scales to make the subjective objective, we don’t have to determine what makes good communication or something more. We have software to do process this, and then use critical skills to figure it out later.

Medgadget: I think that what you are doing is great and really important – what is your philosophy and mission as a company?

Dr. Sahin: Part of our philosophy that is important is that we are trying to bring people together. This is a human-to-human interaction and not a human-to-computer interaction. This is super important. Even though there is a computer involved, it is there to get people interacting with each other more. That’s something that parents ask for and teachers ask for, and what am I saying is that this is actually a two-player game. We have an app that runs on the phone or tablet and an app that runs on Glass. And we’ve made them communicate via our web server. There was a lot of extra infrastructure just have this two-person experience, but it’s super important for the way we’re approaching this.

So in this paradigm, it’s Mom or the teacher sitting there with a phone or tablet who is prompted to say if they are happy or sad or whatever emotion, and the child, young adult, or person using the Glass does what I have described by looking at her and making a determination of the emotion and giving a response. If it matches with the AI found, then you get a point. But more importantly, this platform provides a way for Mom to interact with the child.

A lot of parents tell us that it can sometimes be difficult to interact with their autistic kids, as they are not always aware of whether verbal or non-verbal communication is working. So it can become a sort of charades game with the child, but this would allow a way for the child to get better at understanding a parent’s facial expressions and emotions. Learning things like, yes that’s Mom’s happy face. But in the even higher levels it will involve something like what makes you sad, followed by a little bit of dialogue about war. Or yes, this is what I look like when I’m angry, with a follow-up dialogue what makes me angry. This would be a much more complicated question having to do with the theory of mind and awareness of one’s own self. This is the point at which parents have truly broken down crying, like wow, I haven’t really connected before with my child until now. Like while I have been able to convey emotions though stricter facial expressions, I haven’t been able to really have a dialogue back and forth. My child hasn’t been able to really express what makes him tick or what he thinks or cares about. So this is something extra that has been a part of the experience, and this human element is so important to us.

Medgadget: Who are your partners in development of this Empowered Brain’s technologies? I saw it mentioned that some partnerships include MIT, Harvard, Affectiva, Amazon – how did you get support from these entities? Did you have a personal connection?

Dr. Sahin: My graduate degrees were at MIT and Harvard, and many of the old professors that I had have been very interested in and forthcoming to my innovations. And then I’ve made new connections at each institution, including one to a professor who is a neurologist and is the director of clinical neurotechnology and clinical trials at Harvard Medical School. So that’s been helpful. These are longstanding contacts and respect relationships and mentor relationships that I’ve been developing over the years and decades. Partners like Google and Amazon has been very organic. It started with me buying one Google Glass unit back in the early days and then from there immediately got to know the developer relations person at Glass who put me in touch with the next level up. They really liked the project and invited me to talk on campus at Google, and I have given a few lectures on the Google campus now. It escalated up from there and then the head of the Glass program was personally very interested. So it just kind of worked its way up as something that captures people’s imagination. It is tangible and concrete, and yet it has got lofty goals. I think it’s great to know that Google cares so much about these things, and they really do. They have really helped us.

Also, there’s a side of the government different than the Congress seen on TV that we don’t really know or hear about as often. We got a direct congressional grant that was funded on our work to apply this technology to all the age groups for transition age to get jobs. We have a home-job skills training and job coaching apps that are part of that same congressional grant.

Medgadget: Is there anything else you’d like to share with our readers?

Dr. Sahin: I just want to point out that given that it’s Autism Awareness Month, and we are offering a discounted price for our technology.

What some parents may not know is that our technology would be a valid use of funds from Health Savings Accounts.

We are also open to being contacted by schools and colleges looking to make an impact in this area.

In honor of Autism awareness and acceptance month, and as a result of support from generous benefactors, there are several Empowered Brain systems available at a 50% discount. Availability is limited; systems are available on a first-come, first-served basis. You can order below while this special offer is listed.

Please visit this link: http://www.brain-power.com/purchase/