CRCNS Research Proposal: Measuring and modeling the biological intelligence advantage

As promising as recent artificial intelligence (AI) advances have been, there are still many areas where it falls short. In those areas where it does well, training is slow and requires an unsustainable amount of energy. This has been measured using video games. Humans are able to play these games well after only a few tries, while AI systems require hundreds of thousands to millions of exposures to achieve the same performance. Here, a team of Northwestern University researchers investigate how an animal's ability to complement experience with the use of something similar to imagination underlies the biological learning advantage. In one phase of the work, first-of-its-kind experiments are done with animals evading an autonomous robot threat while large numbers of brain cells are being monitored to decode how the brain is using imagination during avoidance of the robot. In another phase, state of the art methods in machine learning are used to compare to animal performance, while simultaneously building new AI technology to capture the high efficiency learning the researchers observe in biology. With the anticipated need for electricity to fuel AI significantly outpacing infrastructure to provide it, breakthroughs in closing the gap between the energy efficiency of animal intelligence and inefficiency of machine intelligence will be critical to maintain US technological leadership in the coming decades. Leading reinforcement learning algorithms tackle learning using either an explicit model of world (model-based) or use model free methods. Model-based methods appear to perform better in nonstationary environments with partial observability, which is the hallmark of naturalistic scenarios that animal brains have evolved to rapidly master for the sake of survival. Both approaches, however, suffer significant sample efficiency issues. A perspective is emerging that model-based methods may be best for the learning phase to help with sample efficiency, based on results with muZero and with Dreamer v3. At the same time, non-local activity of the hippocampus may be a nexus of memory, planning, and learning, in a manner that suggests a convergent result across artificial and biological systems, where Dyna-style virtual learning substitutes for trial and error in the real world. This perspective, however, leaves open how decision-time planning, in contrast to background planning, finds its niche. The present investigators test the hypothesis that the niche is characterized by high stake contests where partial observability and nonstationarity combine to prevent background planning from being an adaptive strategy. This project is supported jointly by the Neural Systems Cluster in the Directorate for Biological Sciences and the Cognitive Neuroscience Program in the Division of Behavioral and Cognitive Sciences of the Directorate for Social, Behavioral and Economic Sciences. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.