NSF-BSF: Collaborative Research: CIF: Small: Neural Estimation of Statistical Divergences: Theoretical Foundations and Applications to Communication Systems

An abundance of data and recent advances in computation have dramatically increased the reliability and performance of information processing systems. In many applications, these systems provide approximate answers to statistical questions about the distributions that generate the data. Estimators based on neural networks have become the method of choice when dealing with large and complicated datasets. Their popularity is due to their excellent performance in practice and their computational efficiency. But knowledge of the reasons behind their outstanding performance remains quite limited. The goal of this project is to improve the understanding of why neural estimation works and to provide formal performance guarantees that help guide practical applications in fields such as wireless communications. In addition to technological developments, the project features graduate student mentoring, undergraduate inclusion, outreach to high school via a summer camp, international collaboration, and the development of tutorial videos. This project models questions about data using statistical divergences that measure the discrepancy between probability distributions. While there are many approaches to estimating statistical divergences from data, neural estimators have now become the method of choice when dealing with large, high-dimensional datasets. The project will develop a comprehensive statistical and computational theory of neural estimation and apply it to novel applications to communications. It has two main thrusts. The first targets a non-asymptotic neural estimation theory that accounts for all known sources of error: functional approximation, empirical estimation, and optimization. The second will devise a flexible, efficiently computable, and provably accurate methodology for neural capacity estimation and data-driven polar coding. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.