Trustworthy Reinforcement Learning: Inference, Reproducibility, and Adaptivity

In recent years, we have seen a surge of interest in the applications of statistics and machine learning to various problem domains in science and public policy, ranging from healthcare to education to engineering. This growth in the popularity of statistical methods comes with the new fundamental questions. First, most modern machine learning models, including deep learning and, more recently, deep reinforcement learning, are high-dimensional, thereby requiring development of novel statistical approaches for inference for such models. Furthermore, the datasets used in these models are often collected sequentially and, as a result, the data may not satisfy the assumptions of being independent and identically distributed. Examples include data for self-driving cars, online advertising, online recommendation systems, and personalized treatments. This project aims to develop novel statistical inference tools as well as computationally efficient approaches for reinforcement learning in independent non-identically distributed settings, with a particular focus on high-dimensional regimes. The project will offer numerous opportunities for interdisciplinary research training and professional development of the next generation of statisticians and data scientists. The project consists of the two key thrusts at the interface of statistical sciences and reinforcement learning in high-dimensional settings. The first thrust is geared toward statistical inference for reinforcement learning under the high-dimensional scenarios where the data are independent and identically distributed. The focus of the second thrust is to design computationally efficient algorithms, specifically focusing on online and offline reinforcement learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.