CAREER: Learning Hierarchical Generative Models for Expressive, Controllable, and Cross-domain Representations

Data in various domains typically exhibit multiple levels of abstraction, ranging from representations based on language (i.e., semantic representations) to low-level sample-specific details. Examples include in facial images, range from overall facial structure to specific features like eyes and mouth, down to textures like skin and hair, and the layered organization in language data, from topics to paragraphs, sentences, and words. Learning and understanding these data to effectively generate novel content presents a challenge for modern artificial intelligence (AI). Generative AI, which has garnered increased attention in recent years, provides principled methods to begin to address this challenge. Unfortunately, existing models often overlook the structural information within data. They also lack controllability because much of what happens is not transparent. Finally, many models are domain-specific, limiting their capacity in new areas and hindering their use in safety-critical and cross-domain applications. This project aims to go beyond existing generative frameworks by developing new models that exceed their current capacities while maintaining the ease of controllability. The project will also support curriculum development for both graduate and undergraduate programs in artificial intelligence. Furthermore, the principal investigator will continue to mentor undergraduate and graduate students and will be actively involved in pre-college programs for K-12 STEM education. The technical objective of this project is to design new structured generative modeling and learning frameworks to advance existing artificial intelligence for more informative, controllable, and cross-domain representations. Specifically, this project will study and establish core methodological foundations for hierarchical generative modeling in three key thrusts. The first thrust develops context-aware generative models that incorporate the modeling of structural information and contextual dependencies for expressive hierarchical representation. The second thrust develops new structured and semantic-inducing schemes for controllable models. The third thrust develops multimodal generative models for effective cross-domain representations. The comprehensive investigation of the project will lead to the design of more powerful and reliable generative artificial intelligence systems that are easily understandable and effectively manageable by users. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.