Collaborative Research: Random Matrices and Algorithms in High Dimension

Numerical algorithms that can process huge amounts of data are increasingly important, notably for those algorithms used every day within artificial intelligence (AI) software. Examples include voice assistants, facial recognition for cellphones, and machine-learning-based financial fraud detection. But many algorithms are applied only heuristically and remain poorly understood, meaning that theoretical guarantees are missing. In fact, recent AI applications indicate that the direct application of these algorithms, without proper validation, may generate artificial, misleading information. The broad aim of this proposal is to deepen our understanding of classes of statistically-relevant random matrix models that are used to model, analyze and interpret large data sets and to analyze new and classical algorithms as they act on these models. It is expected that this will produce new insights and statistical tools, paired with theoretical guarantees. This award will also train junior researchers and help continue to build the community of researchers working in this field. The proposed problems fit into three main projects. The first concerns the analysis of random matrix models that extend the classical setting of sample covariance matrices. Then by connecting random matrix models and orthogonal polynomials via Riemann--Hilbert problems, the PIs will obtain new estimates and new conclusions about orthogonal polynomials for natural measures generated by these random matrices. Armed with the theoretical results, the second project concerns the direct application of the estimates from the first project, and further refinement of previous analyses, to understand the average-case behavior of numerical algorithms. The focus here is on algorithms from numerical linear algebra. In the third project, the investigators will use the new random matrix estimates, the new results in the theory of orthogonal polynomials and its associated Riemann--Hilbert theory, for both classical and new random matrix ensembles, to generate new algorithms, ultimately leading to new viable statistical estimators. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.