Graph-based complex data analysis

This project develops powerful new tools for understanding today’s most complex data, leveraging cutting-edge artificial intelligence (AI) techniques to help data analysts across diverse fields make informed, automated decisions. Modern science increasingly depends on massive and intricate datasets. Yet many of these datasets are messy, heterogeneous, and too large or irregular for traditional methods to manage effectively. This research introduces novel approaches to analyze such data -- methods that are fast, flexible, explainable, and AI-powered. These tools help scientists and decision makers identify patterns, quantify uncertainty, and make better data-driven decisions. In parallel, the project advances public education in data science and mathematics by creating learning opportunities for high school and college students and by bringing cutting-edge ideas into classrooms and community events. In this way, the project invests in the next generation of talent and underscores the role of AI-enhanced statistical reasoning in solving urgent challenges across science, health, and industry. Technically, the project develops a unified framework for graph-based statistical and machine learning inference in two fundamental classes of complex data: manifold data, which exhibit hidden geometric structures, and mixture data, which arise from overlapping subpopulations. Three core research aims guide the effort. The first develops methods to estimate causal effects in structured data using stochastic nearest-neighbor graphs. The second advances tools to measure statistical dependence and conditional independence using sophisticated graph-based statistics. The third addresses inference in heterogeneous mixture models, with applications to brain single-cell data related to autism spectrum disorder. The project integrates state-of-the-art concepts from statistics, including distribution-free inference, optimal transport, and generative modeling, all of which are central components of today’s AI toolkit. Collaborations with partners from medical and high tech industries help ensure that the resulting methods translate into tools for researchers and professionals working at the frontiers of data science. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.