SaTC: CORE: Medium: Digital Forensics for Deep Neural Networks

The vulnerability of Deep Neural Networks (DNNs) to adversarial attacks remains a major hurdle limiting their use in a wide range of safety-critical applications, such as self-driving cars, medical diagnosis, and financial market analysis. Existing research around adversarial robustness has developed a series of promising defenses, but these are generally broken shortly thereafter by stronger adaptive attacks. This seems unlikely to change, calling for complementary security efforts that increase the cost (in time and computing resources) for successful attacks, reduce the impact of attacks, and identify attackers and vulnerabilities following an attack. This project targets the final goal through developing attack forensics methods to improve the robustness of deep learning systems. The project can alter the playing field between attackers and defenders, by expanding the defense toolset beyond attack prevention to include post-attack analysis. The project will consider three main avenues of attacks on DNNs. The first focuses on analyzing training time poisoning attacks, using clustering and machine unlearning-inspired methods to identify poisoned samples and developing model training methods that trace training provenance to support later forensics. The second focuses on inference time attacks while models are in use, through developing variations of target models that include additional data; the additional data is chosen to maintain accuracy while allowing defenders to make inferences about the models being used to generate attacks. The third focuses on identifying new vulnerabilities arising from neural synthesis models used to generate images, text, and other content, characterizing the space of possible goals attackers might have and methods they might use to poison these models. The project team will also work with industry partners to ground the work and increase the chance of it transferring into real systems, as well as with their department's efforts to increase the number and diversity of people doing research around cybersecurity. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.