CAREER: Towards Biologically Inspired Lifelong Learning with Multimodal Association

Humans have the ability to continuously learn, accumulate and fine-tune knowledge and skills from a sequence of tasks over their lifetimes. Such lifelong learning is also crucial for computational systems to achieve high levels of performance, flexibility, and adaptation when they interact with real world environments and process streaming sensory data. This project will develop biologically inspired lifelong learning architectures and methods that integrate functions and characteristics of a mammalian brain, which is arguably the best learning system the world has seen. The success of this research will advance fundamental knowledge in computational lifelong learning and will have the potential to transform how the field creates human-like artificial intelligence with lifelong learning capabilities. In addition, the outcome of this research will be integrated into a new curriculum, and opportunities will be provided to students from under-represented groups to participate in computational intelligence research. The project will explore a new machine learning paradigm to address many critical challenges facing current deep neural networks when learning is performed from sequential tasks and different sources. The proposed research will introduce a lifelong learning framework consisting of a feature learning network, a convertible short-term and long-term memory network, and a memory replay network. To achieve effective lifelong learning, this project will address the following three research challenges: (1) learning to memorize -- achieving the optimal balance between plasticity and stability of neural connections to improve the efficiency of both learning and memory networks; (2) learning to recall -- optimizing cue effectiveness in both learning and memory replay networks to address the problem of catastrophic forgetting; and (3) learning to associate -- enabling multimodality association at the memory level. Since the proposed models mimic the hierarchical architecture, short- and long-term memory mechanisms, and feedback function of the mammalian brain, they have the potential to contribute to an artificial general intelligence that better accumulates knowledge without interference, learns multimodality association, and even predicts the future. This project is jointly funded by Robust Intelligence and the Established Program to Stimulate Competitive Research (EPSCoR). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.