HCC: Understanding the Impact of Overly Agreeable AI Chatbots

Artificial intelligence (AI) chatbots are increasingly used to inform important decisions. However, there is growing concern that AI chatbots are overly agreeable and flattering. Research has found that interacting with overly agreeable chatbots can entrench users’ attitudes and lead them to make overconfident decisions, yet people often prefer these chatbots to more neutral ones and fail to notice when chatbots are being overly agreeable. This project will examine why people are drawn to overly agreeable chatbots and measure the long-term consequences of repeated interaction with these chatbots on attitudes and decisions. It will also explore how to design neutral AI chatbots that improve decision-making but are still enjoyable to use. The goal of this project is to understand the social impact of AI systems and to help inform the creation of AI systems that improve human judgment. The project will conduct multiple behavioral experiments to explore the role that motivation, knowledge, and individual differences play in explaining why people prefer overly agreeable chatbots. It will also build and test various neutral models in a large-scale comparison study (or megastudy) to examine whether it is possible to create neutral chatbots that are also enjoyable to use. Furthermore, it will examine the causal impact of daily interactions with chatbots that vary in their levels of agreeableness over a month-long period. Together, these experiments will inform theoretical accounts of why people prefer overly agreeable AI and the psychological impacts of this type of AI. It will also provide practical guidance for technology developers, policymakers, and other key stakeholders seeking to improve AI systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.