CAREER: Specification-Guided Imitation Learning

Imitation learning (IL) is a powerful learning paradigm that enables machines, such as robots or artificial intelligence (AI) systems, to learn from demonstrations provided by human experts or expert agents. However, in practice, human demonstrations can be inadequate, partial, imperfect, environment-specific, or suboptimal. To address these challenges, this project introduces a novel framework that allows formal specifications to guide the data-driven learning process of IL. The project’s novelties are new theories and algorithms for combining formal specifications of different forms with data in imitation learning. The project's impacts are manifold, including overcoming the over-reliance on data in current IL approaches, improving their performance and robustness, and catalyzing new cross-disciplinary research at the intersection of machine learning and formal methods. With strong institutional support, the project is providing outreach opportunities to K-12 students and helping inspire younger generations to pursue a college degree or a future career in engineering. The central idea behind this project is that expert inputs need not be limited to demonstrations. For instance, it can be more natural and economical for an expert to specify safety properties that the learned policy must satisfy, instead of showing the many different ways that the system can fail (imagine crashing an expensive vehicle hundreds of times just to generate these "negative examples"). In addition to offering complementary information, specifications can also provide structures to the learning process and improve the overall learning outcomes. For example, automata-based rewards are much more suitable for modeling temporally extended tasks than Markovian rewards. This project is exploring the synergies between data and formal specifications in a unified learning framework, and has the potential to transform how we develop AI systems in the future. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.