Cross-Cutting Improvements:Enhancing Findability and Accessibility of Scientific Data with Large Language Models and Vector Indexing

Scientific research across domains is increasingly driven by large-scale, complex datasets produced by simulations, experiments, and observations. While the volume and variety of data continue to grow, researchers still face major challenges in discovering, understanding, and accessing the data most relevant to their work. This project aims to establish a new data curation infrastructure by integrating Large Language Models (LLMs) and vector indexing into the data life cycle: improving metadata, enhancing discoverability, and optimizing data storage. Existing data management systems primarily depend on structural metadata and keyword-based search, which often fail to capture the scientific meaning, analytical context, or intended use of a dataset. This disconnect significantly hampers data reuse, cross-disciplinary collaboration, and the broader goals of open and reproducible science. This project addresses these limitations by building a new class of data management infrastructure that integrates large language models (LLMs) and vector indexing into the scientific data lifecycle. The framework will introduce an intent-aware semantic layer that enables users to interact with data through natural language, discover relevant datasets based on scientific goals, and guide storage systems in organizing data based on expected use. This work directly advances the FAIR (Findable, Accessible, Interoperable, and Reusable) principles and responds to the priorities outlined in the NSF FAIROS solicitation, which emphasizes sustainable, socio-technical innovations for research data management. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.