CAREER: Improving Grounded Agents via Efficient Feedback from People

Millions of people now use artificial intelligence (AI) systems that carry out tasks on their behalf, from writing computer code to navigating websites to managing data. However, when these systems misunderstand what a person wants, correcting them remains surprisingly difficult. Current methods for providing feedback either don’t provide much information to the system, or are so time-consuming that a person might as well complete the task themselves. This project will develop new methods that allow people to communicate corrections and preferences to AI systems more naturally and efficiently. By making it easier for people to guide and improve AI tools, this research has the potential to make these powerful technologies more practical and accessible to a broader population, including those without specialized technical training. This project addresses the challenge of enabling efficient human feedback for AI agents that operate in complex, long-horizon environments such as code editors and web browsers. The research is organized around three thrusts. The first thrust develops models and benchmarks for agents that can interpret blended feedback -- seamless combinations of natural language instructions and direct user actions such as code edits -- and reason about user intent by considering why a person chose one form of feedback over another. The second thrust creates algorithms that allow agents to generalize feedback by inducing reusable functions that users can inspect, debug, and refine; a single correction to a function can then improve the agent's performance across entire classes of future tasks, reducing the need for repetitive supervision. The third thrust designs agents that proactively ask clarifying questions by generating many candidate solutions in parallel, maintaining a probability distribution over them, and selecting queries that maximize expected information gain while minimizing the cost of interaction for the user. Across all three thrusts, evaluation will combine automatic benchmarks with human studies that measure both task success and interaction efficiency. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.