CIF: Small: Advanced Understanding and Applications of Deep Learning

This project aims to advance Artificial Intelligence (AI) by investigating the mathematical foundations and practical applications of deep learning models, which encompass broad class of neural network methods. The focus is on understanding the properties of neural networks that are trained on large datasets, investigating how these properties enable networks to model complex data distributions and capture intricate dependencies within the data. By analyzing the progressively refined data representations that emerge during training, the project seeks to uncover the principles underlying the ability of neural networks to map and represent complex information. This research promises to enhance our understanding of network architectures and their capabilities, with potential applications in areas such as image processing and natural language understanding. Moreover, the project will train the next generation of AI researchers. Through hands-on involvement in cutting-edge work, students will gain invaluable experience and develop the innovative thinking needed to tackle future challenges. Ultimately, this effort not only builds a skilled workforce but also contributes to the broader goal of advancing AI technology to better interpret and interact with the world. The goal of this project is to deepen the understanding of the deep learning models that underpin modern AI systems by exploring vector-valued, multi-output mappings, compositional function spaces, and the inner workings of transformer architectures. Bridging theoretical insights with practical applications, the project aims to develop novel network architectures, regularization strategies, and training methodologies to improve the performance and generalization capabilities of neural network models. Central to this effort is the study of compositional function spaces and the progressively refined data representations that emerge during training. This includes investigations into both low-dimensional settings, such as implicit neural representations for images and continuous fields, and high-dimensional domains in computer vision and language modeling. A particular focus will be on transformer architectures, with the goal of understanding the evolution of data representations within these models and uncovering task-specific representations enabled by novel training techniques. By examining the function spaces and mappings characteristic of deep learning models, this research seeks to uncover new foundational principles of data representation and processing. These insights promise not only to advance our theoretical understanding of deep learning mechanisms but also to enable innovative AI applications across a wide range of fields. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.