CREATE Methods Core

Project Summary. The Center for Responsible and Equitable AI Technology Evolution for Mental Health (CREATE) Methods Core will make important contributions to support the scalable implementation and delivery of evidence-based psychotherapies (EBPs) for PTSD using Large Language Models (LLMs). The overarching goal of the Methods Core is to serve the Center through 1) shared technical infrastructure and centralized expertise for the Development and refinement of Mental health Adaptive Guidance and Interactive Conversational (MAGIC) tools; 2) Incorporating implementation considerations at the earliest stages of intervention design; 3) Design of rigorous yet flexible research and evaluation methodologies; 4) Developing and testing LLM-based strategies to activate and assess mechanisms of change. Three exploratory projects and a signature project will be supported by the Methods Core LLM development team comprising clinical psychologists, PTSD experts, experts in culturally appropriate EBP delivery for trauma, computer scientists and specialists in AI and human-computer interaction, a health economist, implementation scientists, health service researchers who have implemented PTSD treatments with under-represented clinical populations, a biostatistician, and experts in ethics and digital mental health. We plan to develop best practices and novel strategies for an inclusive, end-user involved development approach based on the Discover, Design, Build, Test (DDBT) framework. We will meaningfully involve administrators, therapists, and patients in the design and build process including opportunities for end-users to refine and test LLMs. Based on our findings and innovations, we will develop materials and resources to support inclusive, equitable, and culturally responsive development and testing of future LLM-based tools, including support for a broader range of EBPs. We will apply and refine a comprehensive Readiness Evaluation for AI Deployment and Implementation (READI) framework to evaluate each CREATE MAGIC tool (including pilot projects) for Safety, Privacy Equity, Effectiveness, Engagement, and Readiness for implementation. Cross-project evaluations will allow us to examine user experience and perspectives, implementation determinants, costs, and feasibility of integration of MAGIC tools into existing workflows. We will provide tools and methodologies to support broader use of the READI framework. Additionally, across projects, we will develop a cutting-edge, LLM-based approach to identify whether key mechanisms of change (such as active learning, cognitive change, behavioral intention, and improved skills) are activated by interaction with the MAGIC tools that we develop and test. CREATE will partner with EBP training programs and mental health systems to test these tools in routine care settings in three exploratory projects and a signature project.