Feedback, Learning, and Robustness in Recurrent Neural Networks: Implications for Human and Artificial Intelligence

A key ingredient in intelligent perception of the world is predictive processing: combining what the senses take in with what a person already knows and expects. As someone listens, the topic, the surrounding words, and knowledge of language help predict what is coming next. This is part of why speech can still be perceived in a noisy room, where much of the sound reaching the ear is muffled or even missing. Combining bottom-up sensory signals with top-down expectations is a hallmark of human intelligence and shapes everyday perception and decision-making, yet we do not understand how the brain achieves it. This project uses computational modeling to evaluate possible mechanisms, using spoken-word recognition as a well-studied example. Because the same balance of evidence and expectation is also central to artificial intelligence (AI), the findings will provide new design principles for more reliable and easier to interpret AI systems, promoting American competitiveness in technology. The project also trains students in computational skills our technical workforce increasingly depends on, while also creating free lesson plans and other materials that will introduce AI to elementary and middle-school classrooms. Researchers who study perception agree that expectation shapes what people hear, but they disagree about how the brain forms and uses those expectations. This project addresses this by building several computational models of spoken-word recognition and comparing them to key signatures of predictive processing in human speech perception. The models differ mainly in how expectations about a partly heard word are fed back from higher levels of interpretation to the earlier sensory processing levels. They include recurrent neural networks used throughout AI, networks with built-in memory mechanisms of various sorts, and predictive-coding models that keep an explicit estimate of the gap between what was expected and what was heard. The results will sharpen theories of how the brain weighs sensory evidence against expectation during perception and decision-making, a foundational principle underlying intelligent behavior. The results will also offer practical guidance for AI design that will make systems more interpretable and possibly more efficient and effective. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.