Collaborative Research: III: Small: Toward Reliable Language Model Systems via Uncertainty Quantification, Unification, and Adaptation

Large language models (LLM) are increasingly used to support multi-step reasoning and decision making in high-stakes domains such as transportation planning and healthcare, yet they often produce unreliable answers while expressing them with high confidence. Confidently presenting erroneous conclusions creates serious risks when such systems guide consequential decisions. This project addresses that problem by developing uncertainty quantification methods that measure, organize, and act on the many forms of uncertainty present in a model's reasoning, so that systems can recognize when they may be wrong and correct themselves. By making automated multi-step reasoning more dependable, the project aims to accelerate the safe adoption of AI technologies in high-risk domains that demand stable and reliable behavior. This project develops new methods for quantifying, unifying, and acting on uncertainty in large language models through three research thrusts. First, it will create accurate and efficient methods to quantify uncertainty in multi-step reasoning by representing a model's reasoning as graphs and applying graph augmentation, capturing structural inconsistencies and intermediate reasoning ambiguities that output-level measures miss. Second, it will develop a framework that unifies the distinct forms of uncertainty a model exhibits, including semantic ambiguity, structural reasoning errors, and inconsistent world knowledge, into a comprehensive characterization of model behavior. Third, it will introduce an uncertainty-aware adaptation method that attributes uncertainty to fine-grained reasoning steps and incorporates these signals into widely used training procedures, including reinforcement learning from human feedback, direct preference optimization, and group relative policy optimization, so that learning prioritizes high-confidence corrections rather than applying a uniform objective across an entire response. Together, these thrusts form a cohesive methodology that advances the science of reliable language model reasoning and provides practical datasets, metrics, and tools for safe deployment in complex, high-stakes applications. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.