Decoding Speech from Electrocorticography

2021 Seed Grant
Joseph G. Makin, Ph.D.
Purdue University

Losing the ability to speak can be devastating. Sometimes, though, people lose this ability without losing the ability to plan the proper mouth movements for speaking. For example, a stroke, or Lou Gehrig’s disease, can spare the parts of the brain responsible for issuing the right “commands” to the lips, jaws, and tongue, while damaging the parts of the nervous system through which those commands would normally travel. In these cases, it’s theoretically possible to restore speech by monitoring the brain cells associated with intended speech, and then using an algorithm that translates those patterns of brain activity directly into the words the person is trying to say.

However, to do that, an electronic monitoring device needs to be implanted on the surface of the brain, under the skull. Fortunately for speech research, some people already get these devices temporarily implanted: patients who are undergoing treatment for epilepsy. In recent work, I developed and implemented a system that translates this brain activity into speech with nearly perfect accuracy–but only when the participant was restricted to speaking just 50 predetermined sentences, using a total vocabulary of 250 words.

Now our lab is proposing to scale this speech decoding up to allow the participant to form any sentence he or she chooses, from a larger vocabulary of 1000 words. To do this, we plan to record brain activity from a group of epilepsy patients for several hours, while they speak sentences from a new set of sentences that are optimized for flexibility. Then we will develop and apply an improved speech-decoding algorithm, incorporating recent advances in “machine translation,” the technology behind (e.g.) Google Translate.

Other Grants

Sarah C. Goetz, Ph.D., Duke University
Uncovering a Novel Role for Primary Cilia in Eph/Ephrin Signaling in Neurons
2022 Seed GrantSarah C. Goetz, Ph.D. Duke University Women’s Council Seed Grant Primary cilia are tiny projections from cells that function like an antenna- they receive and may also send…
Erin M. Gibson, Ph.D., Stanford University
Circadian Regulation of Oligodendroglial Senescence and Metabolomics in Aging
2022 Seed GrantErin M. Gibson, Ph.D.Stanford University The brain consists of two main classes of cells, neurons and glia. Glia make-up more than half of the cells in the brain…
Yvette Fisher, Ph.D., University of California, Berkeley
Dynamic Modulation of Synaptic Plasticity During Spatial Exploration
2022 Seed GrantYvette Fisher, Ph.D.University of California, Berkeley The Virginia (Ginny) & Roger Carlson Seed Grant Cognitive flexibility is critical for appropriately adjusting thoughts and behaviors to meet changing demands…
Byoung Il Bae, Ph.D., University of Connecticut
Unique Vulnerability of Developing Human Cerebral Cortex to Loss of Centrosomal Protein
2022 Seed GrantByoung Il Bae, Ph.D.University of Connecticut Carl & Marilynn Thoma Foundation Seed Grant The cerebral cortex is the largest and outermost part of the human brain. It is…