Probability flows for high-dimensional problems and applications to sampling and generative modeling

Machine learning, and more generally artificial intelligence (AI), is having a transformational impact on science, technology, and our daily lives. Approaches based on deep neural networks have achieved remarkable success in a variety of applications, such as the impressive performance on image, video, and text generation tasks when combined with diffusion models. Despite this, the present architectures lack reliability and performance guarantees. The goal of this project is to develop mathematically sound approaches to some of the relevant sampling tasks in high dimensions, which is typical in the AI setting and where classical computational approaches are unfeasible. The project is motivated by two tasks: one is the sampling of measures given by their density, which is the typical setting in many applications in physical sciences and Bayesian inference, and the other is creating new samples based on a family of examples, namely sampling in the generative setting. The goal is to create efficient approaches with rigorous performance guarantees. Graduate and undergraduate students will be involved in the research of this project, training a new generation of mathematicians who both have knowledge of modern techniques of applied analysis and are cognizant of important questions arising in data science and artificial intelligence. The project will investigate flow-based approaches to sampling that take the view of variational inference. In contrast to popular approaches to sampling where one generates individual samples (e.g. Hamiltonian Monte Carlo sampling), the goal is to flow an initial measure, parameterized in a tractable way, towards the desired target measure. The project will also investigate flows in new geometries on the space of measures, namely the Radon-Wasserstein geometry and its modifications. The aim is to show that the gradient flows of the Kullback-Leibler divergence converge towards the desired target measure and can be approximated well and efficiently by interacting particles in high dimensions. For generative sampling, two approaches will be studied. One is based on modifications of denoising diffusion models that ensure that the reverse flow converges to a close approximation of the true target measure rather than the training samples, thus resolving the memorization issue. The other is based on flows of particle configurations with respect to maximal mean discrepancy (MMD) which can be approximated well in high dimensions by particles. The project will investigate fundamental questions regarding the MMD based models and how accurately both approaches recover the target measure. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.