Collaborative Research: Learning In-Context Improvement for Robotic Systems via Training-Time Failure Simulation

Robots are increasingly present in factories, warehouses, and homes, yet they remain surprisingly brittle. A robot arm tuned to assemble one product cannot adjust when a part changes shape, and a legged robot that walks reliably on pavement stumbles when the ground turns to gravel or a motor wears down. When something unexpected happens, today's robots do not recover or improve — they keep failing in the same way. Improving a learned robot controller requires retraining it, a slow process demanding large amounts of data and expert oversight. Humans work differently: a person fumbling with an unfamiliar tool does not relearn how to use their hands; they notice what went wrong and adjust on the next attempt. This project develops robots that adapt the same way — rapidly correcting behavior within seconds or a few tries, without retraining. The key idea is to deliberately expose robots to failures during training, so they learn the general skill of recovering from errors and can apply it to problems never seen before. The resulting methods promise more reliable robots for applications from manufacturing and delivery to surgical assistance, where machines must adapt to handle new tools, changing terrain, and gradual wear without constant human re-engineering. The project also trains the next generation of engineers: undergraduates and summer research students participate directly in the research, the investigators develop new course modules, and outreach activities introduce a large number of high school students to robotics and machine learning. The underlying scientific problem is that learned robot policies cannot fix their own mistakes: once trained, the behavior of these systems is static. Traditional adaptive control methods overcome exactly this problem — using runtime error feedback to adjust behavior on the fly — but their restrictive modeling assumptions fail in the contact-rich, perception-driven settings where robots operate. This research brings the flexibility of classical adaptive control to expressive, learning-based policies through the development of in-context adaptation algorithms: rather than updating model parameters at deployment, learned components take a history of recent error signals as input, and produce improved behavior through inference alone. Because adaptation is a single forward pass rather than thousands of parameter updates, behavior adapts orders of magnitude faster than with retraining, using only a small amount of data collected online. Such adaptive behavior does not emerge from standard training, where models only ever see successful, in-distribution experience; the project's central insight is that injecting structured failures during training, and supervising recovery from them, forces error-driven adaptation to emerge in the learned system itself. The research instantiates this principle in the three primary objects underlying virtually all robot learning frameworks — control policies, dynamics models, and value functions — developing theoretical foundations and practical algorithms for each. Together, these methods enable a new class of robotic systems: robots that recover from damage and wear, master unfamiliar tools through a few attempts rather than exhaustive pre-training, and continue improving throughout their deployed lifetime. These capabilities are demonstrated on physical robots performing tasks spanning precision assembly, dexterous manipulation, and quadruped locomotion over varied terrain. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.