Collaborative Research: RI: Conditioning Uncertainty on Epistemic Context in Human-AI Communication

Artificial intelligence (AI) systems are increasingly used as collaborative partners in education, healthcare, scientific research, and many other applications. However, when users question an AI system's response, the system often cannot determine whether it has made a mistake, whether the user is uncertain, or whether the conversation has shifted to a new objective. As a result, misunderstandings can accumulate over long interactions, reducing trust and effectiveness. This project will develop the scientific foundations for AI systems that recognize how conversations change the reliability of their own confidence and respond appropriately by clarifying, correcting, or asking follow-up questions. By enabling AI systems to better distinguish between model errors, user uncertainty, and evolving goals, the project will improve the safety, reliability, and transparency of human-AI collaboration. The research will advance fundamental knowledge in trustworthy AI while benefiting applications in education, healthcare, decision support, and scientific discovery. The project will also produce openly available datasets, evaluation benchmarks, and software resources, and provide research and training opportunities for students and the broader workforce in the responsible development and deployment of AI technologies, thereby promoting scientific progress and supporting the nation's technological competitiveness. This project develops a theoretical and empirical framework for modeling how user interaction changes the reliability of AI confidence throughout a conversation. Rather than treating confidence as a static quantity, the project studies how conversational events -including user disagreement, suggestions, and topic shifts - alter the interpretation of model confidence. The research has four objectives: (1) developing a Probably Approximately Correct (PAC) learning theory that quantifies how interaction influences confidence calibration over time; (2) extending this framework to estimate differences between model and user uncertainty; (3) leveraging these estimates to determine when AI systems should clarify, revise, or maintain their responses; and (4) evaluating whether these confidence estimates and interaction strategies remain reliable as conversations become longer and user history accumulates. The methods will be evaluated through coding dialogues with novice programmers, measuring confidence calibration, evolving uncertainty, and dialogue strategies, and through longitudinal studies of repeated interactions. Additional experiments in negotiation and social dialogue will assess generalization across conversational settings. The project will establish a new theoretical foundation for interaction-aware uncertainty estimation and provide algorithms for confidence correction, dialogue planning, and personalization, advancing trustworthy human-AI interaction in domains where uncertainty evolves through ongoing communication. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.