MFAI: Collaborative Research: Generalization of Diffusion and Flow-Based Generative Models via Geometric Perspectives

Powerful generative artificial intelligence (AI) models have emerged in recent years, with applications extending beyond language and images to fields such as drug design and beyond. One fast-growing branch of generative AI models, called diffusion models, is an especially efficient and effective mathematical framework for generative AI. However, despite the strong empirical performance of diffusion models, the fundamental mechanisms that let them generate novel samples remain poorly understood. This Mathematical Foundations of Artificial Intelligence (MFAI) award enables research to develop new theoretical tools to both elucidate how flow-based generative models—a broad framework that includes diffusion models—produce novel outputs and enhance that capability, while also extending these models to handle complex data such as graphs and sets. The resulting theory will strengthen the mathematical foundations of AI and help make the technology safer for real-world use by reducing risks, such as unintentionally copying private training data into public outputs. The project will also nurture the next generation of researchers through student training at the intersection of mathematics and AI. Research enabled by this award investigates two central challenges in flow-based generative modeling: (1) how to achieve controlled generalization to produce diverse and novel in-distribution samples, and (2) how to extend these models to complex data types beyond the Euclidean setting, such as graphs, point clouds, or sets. The first thrust focuses on understanding why trained flow models often generalize better than the theoretically optimal solution suggests, using tools from geometry, ODE, manifold learning, and deep learning theory. The second thrust takes a metric space perspective, formulating a general-purpose meta framework for generative modeling of structured data via geometric tools and optimal transport. New scientific findings are expected to lead to both theoretical insights and new modeling strategies, potentially improving the safety and applicability of generative AI. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.