Collaborative Research: SaTC 2.0: RES: AIGIS: Securing the Deep Learning Model Supply Chain

Pre-trained AI models shared through open online repositories are becoming essential infrastructure for research, industry, and government. But this growing reliance also creates an important cybersecurity concern: just as traditional software can be attacked to include viruses or access backdoors, AI models can also be tampered with. This can lead to security breaches and errors in systems that rely on these pre-trained models. This project will develop methods and tools to help users verify whether a pre-trained AI model is trustworthy before it is incorporated into scientific workflows, operational systems, or other important computing environments. By improving the security of this emerging AI infrastructure, the project will help strengthen the U.S. research enterprise, support economic competitiveness, and improve the resilience of AI-enabled systems. The project will also advance education and workforce development by training students, providing research opportunities, and fostering collaboration among universities, industry, and other stakeholders. This project develops a novel approach to address three major security challenges in the machine learning (ML) model supply chain. The research integrates software engineering principles with machine learning techniques to systematically mitigate vulnerabilities during model selection, loading, and management. First, the team of researchers will tackle model spoofing, where adversaries upload malicious models using deceptive names. The project relies on novel anomaly detection schemes for naming conventions and architectural signatures to identify these threats. Second, the investigators will secure the model deserialization process. Because frameworks often use formats vulnerable to arbitrary code execution, the research will develop automated, least privilege deserialization mechanisms and define safe subsets for model loading runtimes. Third, the project will establish robust model lineage tracking to manage the risks of reusing models. The team will create a lineage graph data structure that combines static and dynamic analysis to trace model evolution and detect illicit modifications. By integrating these methods, the project provides a comprehensive defense system that enhances trust, integrity, and oversight in the open source model ecosystem. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.