CAREER: Towards Continual Learning on Evolving Graphs: from Memorization to Generalization

In the modern big data era, data often grows continuously and its interconnections and temporal dynamics evolve. To cope with the continuous evolution in data, an intelligent agent needs to incrementally acquire, perceive, accumulate, and exploit structural and temporal dynamic knowledge throughout its lifetime. This project aims to develop a generic machine learning paradigm to conduct Continual Learning on Evolving Graphs (CoLEG). The success of this project will 1) benefit critical infrastructure (such as social networks, transportation, and renewable energy) and human welfare (in the form of, for example, improvements in healthcare and epidemiology), 2) provide an ideal platform for composing the areas of graph representation learning, time series analysis, continual learning, and causal analysis, and 3) develop open-source tools for evolving graphs that can advance diverse topics such as node classification, link prediction, and temporal forecasting, improve our knowledge of the physical world, and contribute to real-world applications. This project will also 1) engage high school students in research and outreach to K-12 teachers and students, 2) broaden the participation of students in STEM, and 3) educate undergraduate and graduate students through the development of new course modules in data mining and machine learning. To achieve the above goals, it is imperative to address two core challenges associated with evolving graphs, specifically memorization and generalization. The former aims to facilitate the model in acquiring the capacity to comprehensively retain a vast array of conceivable scenarios, encompassing a comprehensive range of existing structural and temporal dynamic conditions. The latter strives to ensure the model can generalize its knowledge and effectively adapt to unforeseen and complex circumstances. This project will develop a generic machine learning paradigm, CoLEG, to resolve the catastrophic forgetting problem by retaining essential structural information and temporal dynamics, ensure the generalization capability, and address real-world applications on evolving graphs. Specifically, 1) a new continual learning paradigm will be formulated to tackle the catastrophic forgetting issue in structural evolving graphs via graph sparsification and topology-aware embedding, 2) new algorithms will be developed to incorporate structural and temporal dynamic patterns of evolving graphs under different regimes, resolve the task-free challenge, and reveal high-order dependencies, and 3) novel solutions will be constructed to pursue pre-trained models and facilitate test-of-time adaptation to ensure the generalization over unforeseen scenarios. This project will contribute to continual learning, graph representation learning, time series analysis, and domain generalization. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.