CAREER: Foundations of Interactive Machine Learning with Rich Feedback

Distinct from conventional learning paradigms, interactive machine learning captures many real-world settings where learning agents adaptively acquire information and learn to make decisions or gain insights. This paradigm is beneficial in domains where experiments are expensive (e.g., biological imaging and material discovery), as well as in human-safeguarded artificial intelligence (AI) systems (e.g., autonomous driving and chatbots). Despite impressive successes, many critical challenges remain in deploying responsible and resource-efficient interactive learning agents, including a lack of data efficiency, safety, and reusability. The overarching goal of this project is to design principled and practical interactive machine learning algorithms with various feedback modalities that address these challenges, focusing on both single-step and sequential decision-making settings. To this end, the research team will establish theoretical guarantees for the algorithms and release their practical implementations. Integrated with this research project is an education plan that includes an intramural lecture series with the University of Arizona DataLab, a middle school outreach event series in collaboration with the University of Arizona Computer Science Ambassadors Program, as well as undergraduate research projects and curriculum development. The project consists of three interrelated research thrusts. The first thrust studies interactive learning for supervised learning, a single-step decision-making setting. Specifically, the project will focus on designing label-efficient active learning algorithms for nonlinear model classes, including moderately parameterized classes and overparameterized neural networks. The second thrust explores interactive learning for sequential decision-making. It focuses on safe imitation learning from interactive expert demonstrations and interventions, as well as robust reward inference from expert demonstrations. The third thrust aims to address the data reusability challenge in current interactive learning systems by bridging interactive learning and conventional offline learning. Specifically, this thrust will tackle two research questions: first, how to warm-start interactive learners using offline data, and second, how to design interactive learners that collect data amenable to future reuse. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.