He Lost His Voice in 2003. Now, Thanks to a Brain Implant, He’s Getting it Back.

Jul. 20, 2021 News

Neurotechnologies

The Problem: For patients with paralysis or diseases that impact muscle movement such as ALS, Parkinson’s, or cerebral palsy, the lack of effective ways to communicate can be debilitating.

The Study: An artificial intelligence-driven brain implant is helping a paralyzed man translate brain signals into speech. The study, led by NYSCF – Robertson Neuroscience Investigator Alumnus Eddie Chang, MD, of the University of California, San Francisco appears in The New England Journal of Medicine.

Why it Matters: This work demonstrates the power of an exciting new technology to give patients a faster, more seamless way to communicate.

When Pancho was 20, his life was upended. A car crash led to surgery, which led to a brainstem stroke, caused by a post-procedure blood clot. The stroke left him in a coma, and he awoke unable to move.

“I tried to move, but I couldn’t lift a finger, and I tried to talk, but I couldn’t spit out a word,” he wrote to The New York Times. “So, I started to cry, but as I couldn’t make any sound, all I made were some ugly gestures. I wished I didn’t ever come back from the coma I was in.”

His new method of communicating involved a pointer attached to a baseball cap that he used to spell out words letter-by-letter: hardly very efficient.

“I had to bend/lean my head forward, down, and poke a key letter one-by-one to write,” he emailed.

Next he was upgraded to a head-controlled mouse, but this was still labor-intensive. Then, three years ago, he enrolled in a study with Dr. Chang, and everything changed.

A Second Chance at Speech

Dr. Chang’s lab aims to develop ways for patients who have experienced movement loss to speak. Previous work from the team successfully trained an artificial-intelligence-powered system to read signals from someone’s brain as they thought of a word and translate them into written text.

However, this finding was made using subjects who could still speak and involved them first saying the words aloud. The team wasn’t sure if the technology would work for someone whose vocal tract is paralyzed.

“That part of his brain might have been dormant, and we just didn’t know if it would ever really wake up in order for him to speak again,” said Dr. Chang, the Joan and Sanford Weill Chair of Neurological Surgery at UCSF and Jeanne Robertson Distinguished Professor in an article from USCF.

“Our models needed to learn the mapping between complex brain activity patterns and intended speech,” said David Moses, PhD, a postdoctoral engineer who developed the system. “That poses a major challenge when the participant can’t speak. The best way to find out whether this could work was to try it.”

Enter Pancho. He first had electrodes placed in his brain’s speech center that could record electrical activity. He and the researchers then came up with a vocabulary of 50 words – all applicable to his everyday life that could be useful for communicating – things like “water,” “family,” and “good.”

Dr. Chang prepares to connect Pancho’s implant to the computer. Credit: Mike Kai Chen for The New York Times

Then, over 48 sessions, the scientists recorded brain activity as Pancho attempted to say each words – first as short sentences, then as responses to prompts, i.e. “How are you today?” The system used artificial intelligence to predict which word he was thinking of, decoding at a rate of up to 18 words per minute with up to 93 percent accuracy.

While these twice-weekly speech sessions can be exhausting, Pancho is always “looking forward to waking up and getting out of bed every day, and waiting for my UCSF people to arrive.”

“To our knowledge, this is the first successful demonstration of direct decoding of full words from the brain activity of someone who is paralyzed and cannot speak,” remarked Dr. Chang. “It shows strong promise to restore communication by tapping into the brain’s natural speech machinery.”

Importantly, the researchers emphasize that the system isn’t a ‘mind reader’: it’s picking up on signals corresponding to words Pancho is trying to say – not transcribing out his thoughts.

“In the future, we might be able to do what people are thinking,” noted Dr. Chang. “[This raises] some really important questions about the ethics of this kind of technology. [This study, however,] is really just about restoring the individual’s voice.”

A Revolutionary Communication System

For Pancho, the technology is life-changing.

“I just want to, I don’t know, get something good, because I always was told by doctors that I had zero chance to get better,” he typed. “Not to be able to communicate with anyone, to have a normal conversation and express yourself in any way, it’s devastating and very hard to live with.”

During sessions with the electrodes, he wrote, “It’s very much like getting a second chance to talk again.”

The researchers are hopeful that this system will continue to improve to a point where it can be used as a reliable source of fast and efficient communication for patients everywhere.

“It’s now only a matter of years before there will be a clinically useful system that will allow for the restoration of communication,” remarked Leigh Hochberg, MD, PhD, a neurologist with Massachusetts General Hospital, Brown University, and the Department of Veterans Affairs who was not involved in the study.

Pancho also looks forward to when the technology will reach others.

“[In the beginning], I just wanted to somehow be able to do something for myself, even a tiny bit,” he said. “But now I know, I’m not doing it just for myself.”

Diseases & Conditions:

Neurotechnologies

People mentioned:

Edward Chang, MD

2014 NYSCF – Robertson Neuroscience Investigator Alumnus, currently at University of California, San Francisco