SaTC 2.0: RES: Understanding and Detecting Online Scams from Generative AI

Online scams represent a growing threat in cyberspace, exploiting emotional connections, trust, and social vulnerabilities to inflict severe financial losses and long-lasting psychological harm. This project tackles the growing threat of online scams powered by generative artificial intelligence (GenAI) by examining why some individuals are more susceptible than others and identifying protective behaviors that can reduce risk. It also investigates how scammers use AI-generated content, such as fake profiles and messages, and how this affects users. To support users in recognizing and avoiding scams, the project will develop innovative detection tools and explainable warning systems that provide clear and timely alerts during online interactions. The project focuses on the evolving landscape of online scams and develops protective strategies through a coordinated set of research objectives. The project team will examine demographic, psychological, and online behavioral risk factors that increase susceptibility to scams. It will also identify protective factors, guided by Protection Motivation Theory, to help design targeted interventions and prevention strategies that reduce vulnerability to scams. The project team will conduct a longitudinal and multimodal analysis of scammer tactics. This includes detecting the use of GenAI to generate deceptive profiles and communications and assessing the ability to distinguish AI-generated and human-crafted scams through controlled user studies. The project team will develop a context-aware detection and multi-stage warning system powered by large language models. The system will monitor online interactions in real time, detect deceptive patterns across different stages, and deliver explainable alerts to support users in taking timely protective actions. Project outcomes will include open-access datasets of both human-crafted and AI-generated scam content, prototype detection tools for potential integration into online platforms, and educational materials to improve cybersecurity literacy among students, professionals, and the public. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.