CAREER: Mathematical Foundations of Generative Scientific Deep Learning

Generative Artificial Intelligence (AI) has demonstrated its ability to create novel content, such as images and text while also providing tools that are driving breakthroughs in varied scientific disciplines. However, its rapid advancement has introduced fundamental theoretical challenges that remain largely unaddressed. The primary goal of this project is to establish the mathematical foundations of two models that underpin generative AI methodologies in a number of scientific contexts: score-based generative models and transformer-based foundation models. This project will utilize and develop mathematical tools for examining the generative capabilities of score-based generative models in high dimensions and understanding the predictive capabilities and limitations of transformers in solving a broad range of scientific problems. These fundamental understandings are intended to contribute to the development of scientifically reliable AI systems. The project will also support undergraduate and graduate students through research mentorship and education in the mathematical foundations of generative AI. This project aims to study the mathematical underpinnings of score-based generative models and transformer-based foundation models. The project will study the role of fine data structures in mitigating the curse of dimensionality of score-based generative models, through quantifying the improved approximation, statistical and algorithmic complexities in learning high-dimensional distributions with two ubiquitous physical structures: symmetry and hierarchy. The project will also investigate the in-context learning capabilities of transformer-based foundation models for solving partial differential equations by characterizing their scaling laws and generalization performance under distribution shifts. Finally, the project will develop unsupervised foundational generative models for sampling from multiple distributions with provable guarantees. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.