Technology may lead to a way for those unable to physically talk to speak virtually

A pioneering research project at KU may one day give voice to individuals robbed of their speech by neuromuscular diseases such as ALS, also known as Lou Gehrig’s disease. 

Led by Jonathan Brumberg, the project is an attempt to use technology to bridge the gap between people who know how to speak but cannot move the jaw, lips, and tongue needed to produce that speech.

To get there, Brumberg, assistant professor of speech-language-hearing, and his team are recruiting people diagnosed with disorders such as ALS or brain-stem stroke with locked-in syndrome to study what parts of the brain are involved in speaking. The research will then use those signals to control a “virtual” vocal tract computer simulation. This would allow someone to control the jaw, lips, and tongue of a computerized vocal tract in order to create speech.

In Brumberg’s laboratory, study participants wear an electrode cap while they imagine producing speech. This cap, which looks like a swim cap, has 64 electrodes in it that measure brain activity. The cap and electrodes help identify what parts of the brain are being used when participants, who cannot talk or have difficulty talking, are asked to imagine creating speech.

"To start the training, we work with letter combinations such as 'ama' and 'ana,'" Brumberg said. "This is similar to 'b' and 'd' babbling that babies do except that 'm' and 'n' are nasalized versions. Then as participants get good at the task, they'll switch to 'amu' and 'ani.' The 'u' uses the lips, like 'm' does, and 'i' uses the front of the tongue like 'n' does."

The overarching goal of this work is to allow people to communicate using the virtual vocal tract at a conversational speed, but Brumberg says that would be very far in the future. It would take months or even years to teach the patients how to use this technology in order to effectively communicate. Right now, he and his team are focused on producing simple consonants and vowels.

One of the rewarding aspects of the research, said Brumberg, is working with the study participants. The participants donate eight weeks of their time coming to over fifteen sessions with Brumberg and his team. The average life span of someone diagnosed with ALS is five years.

“They are all enthusiastic,” Brumberg said. “For some of them, they have a disease for which there is no cure, and they know that, but this is something they can be a part of. Being a part of it now actually could make a difference later.”