Collaborative Research: III: Medium: Neural Networks Meet Bandits: Theory, Algorithms and Applications

Deep learning techniques have achieved good success in the perception tasks such as image classification, where the correct model output can be obtained and annotated beforehand. For example, when a deep learning model is asked to identify dogs and is trained at the start with a series of images of dogs with annotations identifying dog features (e.g., tails, fur, muzzles etc.), these models do well. However, when the deep learning models are deployed to support decision making tasks such as medical diagnosis, autonomous driving, and conversational systems, only incomplete feedback for training is available. When the correct outputs in decision-making situations are not available to train a system, it is difficult to employ traditional deep learning techniques directly. Bandit learning methods are algorithms that have been developed to deal with incomplete feedback information for learning. This is important because previous work in the perception area, could afford to blindly optimize a model for the best response. For decision-making, this type of optimization may not be optimal, because small mistakes in perception can lead to huge losses in quality of decision making of which the model may not be aware. This project aims to bridge the gap by training the deep neural networks in their natural use context to directly optimize decision making. The goal is to develop a suite of neural bandit learning algorithms, which leverage the most recent advances in deep learning theory for provably efficient neural network model training with bandit feedback. The project consists of three research thrusts. Thrust one develops bandit learning methods in more advanced neural network architectures and studies new deep learning theory with bandit feedback. Thrust two investigates neural bandit learning in decentralized and distributed settings. Thrust three equips the learnt models with privacy and adversarial robustness guarantees. The team of researchers will develop an open-source neural bandit library and teaching materials to disseminate research outcomes and make them publicly available to the broader community to benefit research and education. The project provides unique training opportunities in machine learning and artificial intelligence for undergraduates and graduate students, especially students from underrepresented groups. The researchers will also engage K-12 students, fostering their interest in STEM by educating them on deep learning and AI techniques. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.