CAREER: Data Attribution and Curation for Web-Scale AI Systems

Artificial intelligence (AI) systems increasingly influence how people search for information, receive recommendations, learn, work, and create. The quality of these systems depends heavily on the data used to train them. However, modern training data is often enormous, noisy, and constantly changing, because it is drawn from many sources. Poorly understood data can reduce accuracy, amplify harmful information, weaken reasoning and/or make it difficult to recognize the value of content in AI systems. This project develops new tools for understanding how individual components of training data affect the behavior of large AI systems. These tools will help researchers and practitioners decide what data to keep, remove, prioritize or compensate for. By making data curation more principled and transparent, the project has the potential to improve the performance, reliability, and safety of widely used technologies such as language models and recommendation systems. The project will also create public educational resources, open-source software, conference tutorials, course modules, and research opportunities for graduate, undergraduate, and pre-college students, helping widen access to data-centered AI research and training. The technical goal of this project is to advance data attribution, a family of methods that estimate how training examples influence model behavior. The project will address three major gaps that limit current attribution methods in production-scale systems. First, it will develop a framework for attribution when training data is non-stationary, meaning that future data depends on the evolving model. This framework will estimate long-term cascading effects in settings such as online reinforcement learning, synthetic data generation for self-improving language models, and user feedback loops. Second, it will design attribution methods for complex objectives and architectures, including objectives such as safety, alignment, novelty, and reasoning that are not well represented by a single accuracy number; training processes that use modern optimization algorithms; and modular models such as Mixture-of-Experts architectures. Third, the project will test and refine the methods in industry-relevant settings, including video recommendation, proportional content attribution, and training-data curation for reasoning models. The resulting algorithms will be integrated into open-source software and evaluated through controlled experiments, benchmark studies, and real-world collaborations. Together, these activities will move data attribution from simplified research settings toward practical tools for auditing, improving, and governing large-scale AI systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.