Collaborative Research: SaTC: CORE: Medium: Socio-Technical Interventions Against AI-Generated Abuse

The use of AI for abuse against persons is a growing threat. Decades of research in computer vision and AI has led to the broad availability of tools that can be misused to distort an image of a person, such as turning a clothed image of the individual into an unclothed image without the person’s consent. Attackers profit from this kind of abuse and use these abuses in a range of cyberattacks including blackmail, extortion, and ransomware. To address this problem, the research team is conducting technical and content analysis of the ecosystem of tools used to create abuse material, leveraging principles from decades of psychological research to deter attackers from creating and viewing this material, and engaging in technical forecasting to predict and stop future attacks. Overall, this project aims to protect Americans by technically characterizing the current and future ecosystem of online abuse attacks and developing sociotechnical mitigations against such attacks. This team takes a defense-in-depth approach to abuse reduction. The project aims to make current and future abuse tools harder to access and use and to decrease attacker’s inclination to use those technologies. They combine approaches from internet measurement and content analysis to scope the severity of AI use in abuse and identify signals that can be used to proactively remove abuse tools and content. They also rely on decades of psychological and communications theory and research to develop effective deterrence systems that target attacker motivations and are delivered at the time of action as a form of secondary intervention. They use a series of computational and human-subjects experiments to upper- and lower-bound future attacks that use AI for online abuse. Together, these efforts will develop a robust set of threat models, forecasts and interventions against online abuse. -- This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.