Collaborative Research: BLoG: A Bi-Level Optimization Framework for Learning Over Graphs

Graphs, representing complex sensing and other societal systems like disease networks, social networks, and communication networks, are essential in understanding interactions within these systems. By accurately modeling relationships and structures within data via graphs, today machine learning over graphs (LoGs) plays a vital role in various applications. However, LoG introduces additional hyperparameters such as graph topologies and nodal embeddings into the already complicated neural network training processes. Traditionally, LoG approaches relied on user-defined heuristics to extract features encoding structural information about a graph. However, this process becomes prohibitively expensive in large models and high-dimensional data regimes, and the performance of LoGs highly depends on the choice of these hyperparameters. To address these challenges, the project puts forth a unified bi-level optimization-based training framework for LoGs with automatic selection of hyperparameters. The project also supports the education and diversity goals of the NSF by integrating LoGs research advances into machine learning courses taught in University of California at Irvine and Rensselaer Polytechnic Institute, making cutting-edge LoGs techniques more accessible to a wider range of researchers and students, fostering innovation and inclusivity in the scientific community. Towards this goal, this project aims to develop a bi-level optimization (BLO) framework for trustworthy and efficient LoG, called BLoG. In addition to the basic algorithm and optimization theory development for BLoG, the project will build a tri-level BLoG problem for robust and adversarial graph neural network training tasks, tailoring gradient-based BLO algorithms to these problems. The project will also develop a BLoG framework with multiple lower-level problems for multiple LoG tasks, named Fast-BLoG. Fast-BLoG will tackle fast and efficient semi-supervised graph neural network training. The project will highlight the advantages and new technical challenges of using the BLoG framework for handling machine learning tasks over graphs. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.