Collaborative Research: Intentional Stance Learning of Belief Dynamics, Objectives, and Control Policies in Autonomous Agents

This NSF project aims to create a new mathematical and learning framework for interpreting and predicting how autonomous agents (people, animals, or machines) behave, from what they do, not how they function internally. It is built on the intentional stance, an idea rooted in philosophy-of-mind that explains behavior by attributing goals and beliefs to an agent, even as an imperfect or sub-optimal one. The project will bring transformative changes to how engineers model complex, adaptive, and imperfect decision-makers, moving beyond methods that assume that agents act optimally or that their inner workings are known. This will be achieved by combining machine learning, control theory, and neuroscience to jointly infer an agent's evolving beliefs, goals, and strategies, while guaranteeing the models stay stable and reliable. The intellectual merit of the project includes a mathematical and algorithmic formulation of the intentional stance that learns time-varying beliefs, objectives, and control strategies with formal stability guarantees. The broader impacts of the project include next-generation assistive devices, such as exoskeletons that adapt to a user's intent, which will improve quality of life for millions affected by walking impairments. Broader impacts also include new courses, undergraduate research training, open datasets, and outreach that help attract middle- and high-school students to STEM. Existing methods for interpreting behavior have limitations. Inverse Reinforcement Learning, which infers an agent's hidden objectives, usually casts decisions as Markov Decision Processes suited to high-level planning but not the continuous, moment-to-moment control that real autonomous systems require. Overcoming this limitation demands complex nested computation that adds delays and errors. Inverse Optimal Control handles continuous control while recovering the cost a controller minimizes but requires known dynamics and agent optimality. Yet real agents routinely violate these assumptions. This project develops Dynamic Deep Predictive-Intentional Stance Learning (DDP-ISL), which introduces time-varying belief dynamics as proxies to explain observed behaviors as if they were optimal, and to jointly infer the agent's dynamic objective and control policy, rather than requiring optimality or known dynamics. These models will be learned using Lyapunov-based deep neural networks with real-time training algorithms that guarantee training stability. Inspired by the neuroscience free-energy principle, they will use their predictions to continually self-refine, improving adaptability and robustness. Two settings are addressed: one with observed states and control inputs observed, while the other one with only observed states. DDP-ISL, in both settings, will be applied to exoskeletons for individuals with gait asymmetries to drive the impaired leg to track the healthier leg's behavior and learn its intent. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.