Collaborative Research: MFAI: Combinatorial and topological modeling of trade-offs in language generation: Breadth, validity, and plasticity

The aim of this project is to investigate a minimal mathematical framework known as “language generation in the limit” in order to build a rigorous, foundational science of generative language learning. Generative AI and large language models (LLMs) are having a huge impact on the economy, on scientific research, and on society, yet a solid grasp of the mathematics behind why they actually work or what problems they are capable of solving is still lacking. Because these models operate essentially as black boxes, they struggle with a fundamental trade-off between validity and breadth: there is always the risk that might either safely repeat a narrow set of known facts, or might generate creative but fabricated information, in a process commonly known as hallucination. This project serves the national interest and promotes the progress of science in this area by abstracting away from the complex engineering issues surrounding these systems to build a concrete mathematical foundation for language generation. By reducing the problem to its theoretical core, hard limits can be established on what AI can and cannot do. Ultimately, these theoretical insights can provide part of the groundwork needed to accurately evaluate and manage AI's capabilities and limitations, contributing to the broader goal of making these technologies more interpretable and reliable, all while training a new generation of STEM students to tackle the mathematical challenges of emerging AI systems. The approach followed in this project allows the study of AI language generation to proceed entirely independent of any specific implementation instead of relying on the specific architectures or data distributions of current models, which are constantly evolving. The project develops quantitative density metrics to measure the capacity of models to achieve broad, correct generation, alongside novel topological methods to characterize exactly when high-density generation is possible and when it faces provable combinatorial obstructions. By bridging topology, combinatorics, analysis, and learning theory, the research provides a unified theoretical foundation for evaluating the core capabilities and fundamental limitations of generative systems. Beyond producing foundational theorems for AI reliability, this work fosters an interdisciplinary research community through open-access resources and student mentorship. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.