CAREER: Frugal Learning of Reliable Dexterous Manipulation Skills

To effectively operate in a complex world that is designed by and for humans, robots require reliable and adaptable dexterous manipulation skills. Indeed, such skills will enable robots to revolutionize a wide range of domains, such as agriculture, assistive robotics, manufacturing, warehouse automation, robotic surgery, military, and consumer robotics. While modern deep learning-based approaches to dexterous manipulation are starting to demonstrate impressive capabilities, they often rely on significant technical expertise and require vast amounts of data and computing resources that are prohibitive outside of well-resourced laboratories. This Faculty Early Career Development (CAREER) project aims to develop efficient computational methods capable of learning robust dexterous manipulation skills without relying on significant user expertise or vast amounts of data and computing resources. To this end, this project will develop innovative and efficient approaches to learn reliable dexterous manipulation skills that offer significant improvements in three important dimensions: efficiency, self-sufficiency, and reliability. As such, this project will reduce technical and financial barriers to entry and enable the wide-spread adoption of dexterous manipulation in several critical domains. Further, the project will deliver insights and open-source technologies with broad relevance to many robotics problems involving high-dimensional systems and highly nonlinear dynamics (e.g., legged locomotion). The project includes a multi-faceted education and outreach program to educate high school students in some of the most rapidly-growing areas of science and expose them to careers in research and academia. This project will also establish and strengthen ties to neighboring Historically Black Colleges and Universities and Minority Serving Institutions via a shared seminar series and a student research exchange program. Despite decades of rigorous development of control methods and significant recent strides fueled by deep learning, achieving reliable autonomous dexterous manipulation remains challenging, with human-level dexterity still an elusive target. This state of affairs is likely due to three often-overlooked aspects of existing methods: i) high barriers to entry due to demands for expensive computational resources and annotated data, ii) inability to handle new tasks without relying on significant user expertise (e.g., for reward or controller design, hyperparameter tuning, and data annotation), and iii) unreliable behaviors due to inscrutable and unpredictable learned policies. The overall objective of this project is to enable robots to learn reliable dexterous manipulation skills without relying on significant human effort and computational resources. The project will tackle its objective’s fundamental challenges by combining the computational efficiency and reliability of operator theoretic tools from dynamical systems theory with the self-sufficiency and expressivity of unsupervised representation learning. The project will develop algorithms to i) learn dexterous manipulation skills from observations, ii) learn multi-sensory representations and models from self-guided play, and iii) analyze and guarantee learned manipulation skills. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.