EAGER: NAIRR Pilot: Applying NAIRR Pilot Resources to Optimize AI-Ready Data Investments

This EAGER project will demonstrate how National Artificial Intelligence Research Resource (NAIRR) Pilot resources can facilitate the development of a new service to the cybersecurity research community, specifically to infer the utility of datasets and software tools based on their documented use in scientific publications. The service will enable the assessment of scientific data utility to address a growing need in the scientific research community: optimizing AI-ready data investments. The research will help funding agencies optimize investments in AI-ready datasets, provide a scalable model for other similar scientific disciplines, and enhance the broader research community's ability to evaluate and sustain valuable data resources. State-of-the-art techniques for automatically extracting links between resources cannot handle the complexity, variability, and contextual nuances of natural language, e.g., in discerning a reference to a resource from its use in a paper. This project will require data collection, preprocessing and labeling, model selection and training, as well as evaluation and deployment. The approach will leverage techniques such as prompt engineering and fine-tuning to extract relevant metadata from unstructured text, and associated inference of relevant annotations. The research will utilize multiple NAIRR compute resources and aims to provide a repeatable, AI-based method for inferring the utility of datasets and software tools. This capability will enable more accurate interpretation and explanation of complex relationships in scientific publications. The resulting service will address a persistent and growing cybersecurity research challenge, and serve as a model for other disciplines. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.