EAGER: Solving Representation Learning and Catastrophic Forgetting with Adaptive Resonance Theory

Improving representation learning is a key challenge for improved machine learning models, particularly for online and real-time applications. Current machine learning methods are expensive and subject to catastrophic forgetting by overwriting previously learned information. Adaptive Resonance Theory (ART) is designed to avoid this overwriting, but it accomplishes this by combining match-based learning with a reset mechanism. This combination is effective at avoiding overwriting at the cost of missing the representation learning capabilities common in gradient approaches such as deep learning. Intellectual Merit: This project seeks to combine the advantages of match- and error-based learning by using first principles in combined design, as opposed to ad-hoc hybrids. It will investigate the potential for overcoming scalability and credit assignment problems entailed in hybrid approaches to date. It also seeks to improve explainability of ART by appropriate design of the resulting templates formed during training. It will also clarify the difference between catastrophic forgetting due to overwriting correctly learned information, versus apparent but valid performance degradation due to new information that was not represented during previous training. The goal is to solve the former and learn from the latter. Broader Impacts: The project will produce technological broader impacts of more cost-effective and trustworthy machine learning. The resulting systems will be less brittle and more explainable. They will not need an artificial freezing of learning capability but will be true lifelong learning machines. Demographic broader impacts will include mentoring for prospective researchers and undergraduates. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.