HCC: Human-AI Teaming for Health Coaching

More people now rely on artificial intelligence (AI) conversational agents (chatbots) for advice on their health, wellness, and nutrition. In the world of nutrition, these AI chatbots may expand access to experts. However, AI chatbots may give confusing, incorrect, or even harmful advice. Working together, health coaches and AI tools may make digital nutrition coaching safer, more reliable, and more helpful. Today, platforms exist that enable experts to create AI chatbots by describing how they should behave. However, these tools are designed mainly for tech savvy users, such as software developers, making them difficult for health coaches to use. They also focus on the needs of experts alone, even though a digital coach should reflect a health professional’s expertise while adapting to each client's goals and preferences. In this project, we will focus specifically on chatbots that rely on Large Language Models (LLM) and will develop PACT (Participatory Alignment for AI-Empowered Coaching Teams), a platform that enables dietitians without technical expertise to create and manage AI/LLM coaching chatbots. PACT will allow the team to work together to shape the AI/LLM chatbot’s behavior, while providing dietitians tools to monitor AI/LLM recommendations and intervene when needed to keep the AI/LLM chatbot’s advice safe and appropriate. Human-AI teaming solutions allow users and designers to leverage large language models (LLMs) for complex behaviors by introducing complex workflows and multi-agent architectures. Advantages of human-AI teaming include greater transparency, an ability to rely on human expertise to guide LLM performance, and greater user control. At present, however, the vast majority of tools for supporting human-AI teaming focus on technically savvy users, such as software developers. The applicability of these solutions for users from other non-technical domains remains underexplored. In this project, we explore Human-AI teaming in the context of hybrid health coaching, where a human expert and a digital chatbot work together to provide coherent, consistent, and continuing coaching. Crucially, for such hybrid coaching to be safe and effective, AI chatbots must work in union with human coaches, who specify the requirements and constraints to govern AI chatbot actions. We propose to achieve this through a rigorous, three-phase research agenda centered on the development and evaluation of a novel Human-AI Teaming platform, named PACT (Participatory Alignment for AI-Empowered Coaching Teams). PACT will help registered dietitians and their clients to achieve three distinct goals. First, it will allow them to articulate specifications and needs for AI chatbots in an intuitive way and automatically translate these specifications into a multi-step workflow and multi-agent architecture. Second, it will help them to negotiate and resolve potential conflicts between chatbots and clients. Finally, it will allow dietitians to monitor AI chatbot adherence to the human coaching specifications. Our research is organized into three synergistic thrusts, including Thread 1 for participatory user-centered design activities, Thread 2 for development and validation of PACT, and Thread 3 for a two-arm evaluation study focused on usability and shared decision making, robustness and reliability of AI chatbots, and interaction impact. This project complements its technical agenda with a complementary set of educational activities that include research opportunities for undergraduate and high school students, a hack day for undergraduate students at Columbia University, and a set of lectures on Human-Centered AI for medical students at the Columbia University School of Physicians and Surgeons. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.