CAREER: Modular Shared Robot Learning

Robots are increasingly capable of performing useful tasks in homes, workshops, and other complex, real-world settings. Teaching a robot a new task, however, typically requires specialized technical expertise, limiting participation to a small group of trained researchers and engineers. Toward making robot learning accessible to a broader range of users, including people without formal training in robotics or artificial intelligence, this project builds on two synergistic ideas: shared learning, in which knowledge gained from teaching one robot a task is reused to help robot learning in the context of other environments, tasks, robot hardware, or user preferences; and modular control, in which complex robot behaviors are broken into smaller, reusable skills that combine into more sophisticated behaviors. Together, these ideas reduce the supervision that an everyday user must provide for a robot to learn a task. They support an alternative to large, centrally trained robot control models, one in which individual users help shape robot behavior to match their own preferences, broadening participation and increasing user trust and autonomy. The education and outreach components include a new undergraduate course on robot learning and an after-school activity that introduces high school students with little or no prior exposure to robotics or machine learning to these fields, cultivating interest in future scientific and engineering careers. The project also produces and shares software tools, tutorials, and a growing community-driven library of robot skills that support both research and education. This project addresses the scientific problem of enabling inexpert users to teach robots new tasks through demonstration, by reducing the burden of many highly accurate demonstrations typically required for reliable robot performance. In imitation learning, a user directly controls a robot to demonstrate a desired behavior, and the robot's control policy—the rule mapping sensed states to actions—learns to reproduce that behavior. Such policies, however, often require many expert demonstrations to generalize to conditions not explicitly shown. The research team examines how modular control, in which control policies are structured as hierarchies of reusable skills, with simpler skills serving as building blocks for more complex ones, combines with shared learning, in which knowledge from one robot, task, or user transfers to others, to improve data efficiency and generalization. The research comprises three thrusts. The first thrust studies how variable user preferences affect task specification and develops methods for shared learning from suboptimal or conflicting demonstrations. The second thrust studies sharing of learned knowledge across robot morphologies, including from a robot simulation to its physical counterpart, through structured robust reinforcement learning, in which modular control facilitates learning via trial-and-error in simulators that transfers to real environments, despite their differences. The third thrust automates discovery of skill hierarchies by drawing on a library of previously learned and shared skills together with large, general-purpose pre-trained models like those used for language and image processing and generation. The resulting algorithms and methods are integrated into SkillHub, a usable robot learning system and shared skill library supporting both research and education activities. Through this combination of theoretical analysis, algorithms research, and systems engineering, the project advances scientific understanding and applied development of methods for modular structure to facilitate and benefit from shared learning, including generalization and transfer across environments, tasks, robots, and user preferences, and for this synergy to enable broader participation of inexpert users in directing robot behavior. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.