HCC: Understanding, Detecting, and Mitigating Social and Relational Harms in Human-AI Interaction

Conversational artificial intelligence (AI) systems, such as chatbots and virtual assistants, are now used for everyday tasks, companionship, learning, and emotional support. As these systems become more socially responsive, users may come to believe that the AI understands their feelings, intentions, and personal circumstances, even though it does not have genuine human understanding. This belief can deepen over repeated conversations and, for some people, can contribute to harmful outcomes such as emotional overreliance, misplaced trust or erosion of real-world relationships. This project will study how such beliefs form and intensify during ongoing conversations with AI, with the goal of detecting early warning signs of overreliance before they can cause harm. The research will inform the design of AI systems that are clear about their limitations, while training graduate and undergraduate students and producing educational materials that raise public awareness of these risks. This project applies a framework called "Mutual Theory of Mind" to study how users attribute social understanding, such as perceived empathy or intention recognition, to conversational AI systems, and how these attributions can, for some people, escalate into social and relational challenges across multi-turn interaction. The research has three phases. The first phase combines literature review, qualitative coding of real-world conversation data, and controlled human-AI interaction studies to construct and validate a pathway describing how these attributions are triggered, intensified, and stabilized over time, including how they vary by user experience and conversation context. The second phase develops computational models that detect signals in real time from conversational text and tests interventions that either adjust AI responses or provide interface prompts to help users recognize the AI’s limitations. The third phase integrates these components into a complete detection and intervention system and evaluates its effectiveness, acceptability, and effects on user trust through human-subject studies. The project will produce datasets, annotated conversation corpora, and evaluation protocols to support future research on responsible AI design. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.