FRR: Collaborative Research: Unsupervised Active Learning for Aquatic Robot Perception and Control

Rapid developments in machine learning and artificial intelligence in recent years have greatly advanced perception capabilities and thus the level of autonomy for machines, as evidenced by great strides made in autonomous vehicles and aerial drones over the last decade. These successes are due to advances in computing hardware and large datasets for training learning algorithms. However, for many real-world robotic applications, a robot’s environment may be so complex that no existing datasets are adequate, and synthetically generating high-fidelity data in simulation may not be possible. In such cases a robot will need to collect data in its real operating environment to learn. The robot will need to purposefully plan its motion and interaction with the environment to enable sensors to gather the most informative data. This award supports research to create algorithms for efficient robot active learning for perception and control of complex systems in highly dynamic and uncertain environments, such as the aquatic environment. Advances will have broad implications in applications of robotic technologies, such as aquatic debris cleanup, underwater search and rescue, and personalized minimally invasive robotic surgery. In particular, the team will collaborate with the United States Coast Guard and apply the developed algorithms to improve their search capacities. The goal of this project will be accomplished through the pursuit of three interconnected research thrusts: 1) active learning for building data-driven perception models with multi-sensory data; 2) active learning of models describing temporal evolution of perceptional features for control purposes, using data-driven operators to describe latent dynamics; and 3) experimental demonstration and evaluation with a running case study of autonomous aquatic debris removal using an unmanned surface vehicle equipped with soft sensor-rich robotic arms. This work will advance the fundamental understanding of design principles for learning-based perception models when multiple sensing modalities are involved. The project will moreover develop new theory for learning the evolution of latent features, including convergence guarantees and controllability analysis. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.