Decoding Speech from Electrocorticography

2021 Seed Grant
Joseph G. Makin, Ph.D.
Purdue University

Losing the ability to speak can be devastating. Sometimes, though, people lose this ability without losing the ability to plan the proper mouth movements for speaking. For example, a stroke, or Lou Gehrig’s disease, can spare the parts of the brain responsible for issuing the right “commands” to the lips, jaws, and tongue, while damaging the parts of the nervous system through which those commands would normally travel. In these cases, it’s theoretically possible to restore speech by monitoring the brain cells associated with intended speech, and then using an algorithm that translates those patterns of brain activity directly into the words the person is trying to say.

However, to do that, an electronic monitoring device needs to be implanted on the surface of the brain, under the skull. Fortunately for speech research, some people already get these devices temporarily implanted: patients who are undergoing treatment for epilepsy. In recent work, Dr. Makin developed and implemented a system that translates this brain activity into speech with nearly perfect accuracy–but only when the participant was restricted to speaking just 50 predetermined sentences, using a total vocabulary of 250 words.

Now Dr. Makin’s lab is proposing to scale this speech decoding up to allow the participant to form any sentence he or she chooses, from a larger vocabulary of 1000 words. To do this, they plan to record brain activity from a group of epilepsy patients for several hours, while they speak sentences from a new set of sentences that are optimized for flexibility. Then they will develop and apply an improved speech-decoding algorithm, incorporating recent advances in “machine translation,” the technology behind (e.g.) Google Translate.

Other Grants

Rebekah C. Evans, Ph.D., Georgetown University
In Vivo and Ex Vivo Dissection of Midbrain Neuron Activity During Exercise
Exercise is important for the health of the body and the mind. Exercise promotes learning and reduces symptoms of brain-related diseases such as Parkinson’s disease and Alzheimer’s disease. However, it…
William J. Giardino, Ph.D. Stanford University
Deciphering the Neuropeptide Circuitry of Emotional Arousal in Narcolepsy
This research project aims to investigate the neural mechanisms of a specific type of brain cell called neuropeptide neurons within a region of the brain’s amygdala network called the bed…
Howard Gritton, Ph.D., University of Illinois
Attention Mechanisms Contributing to Auditory Spatial Processing.
Our world is composed of a rich mixture of sounds. We often process sounds including speech in the presence of many other competing auditory stimuli (e.g., voices in a crowded…
Nora Kory, Ph.D., Harvard University
Elucidating the Fates and Functions of Lactate in the Brain
The human brain requires significant energy to function. Despite accounting for only 2% of our body weight, the brain consumes a substantial 20% of the body’s energy, relying on a…