Collaborative Research: Online Statistical Inference for Modern Machine Learning

This research aims to develop statistical tools to improve the reliability of artificial intelligence (AI) that is widely used in real-world systems such as automated decision-making, financial forecasting, and neuroscience research. Modern AI often relies on efficient machine learning algorithms to process large-scale, sequentially arriving datasets. While these algorithms are powerful, understanding their behavior and measuring their uncertainty remains a major scientific challenge. To bridge this gap, the investigators will focus on establishing mathematically rigorous methods for uncertainty quantification to build trustworthy AI. Applications will include enhancing theoretical guarantees and interpretability of neural networks, providing robust estimation and inference for econometric and biomedical studies, and detecting real-time change-points in high-dimensional time series data. The projects will promote the progress of science through open-source software and graduate education, and will support the national interest by contributing to reliable, data-driven decision-making in fields important to economic resilience, public health and national security. This research will provide a comprehensive theoretical framework for online statistical inference in machine learning, focusing on constant learning-rate stochastic gradient descent (SGD) algorithms. It addresses fundamental challenges such as non-stationarity caused by arbitrarily fixed initialization and complex dependency structures arising in recursive estimation. The investigators will derive the limiting distributions of SGD-type estimators and construct confidence regions with guaranteed asymptotic coverage. Specific efforts will include (1) establishing Gaussian approximations for high-dimensional dropout regularization, (2) deriving limiting distributions for SGD under non-smooth quantile loss functions using characteristic function techniques, and (3) developing online inference procedures for quantile change-point detection in high-dimensional time series using a novel Bahadur representation. These methods will be supported by numerical experiments and implemented in publicly available software. The results shall provide foundational advances for statistical inference in modern machine learning, bridging theoretical developments with practical applications in dynamic, high-dimensional environments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.