Automated Coding of Exposure Therapy Quality using Natural Language Processing

Project Summary Quality monitoring is essential in psychotherapy clinical trials and for supporting the use of evidence-based interventions (EBI) in mental health service settings, but few are available, and these require costly and laborious procedures. Without pragmatic, specific measures of quality, it is impossible to know whether key quality components are being delivered, and to what effect. Natural Language Processing (NLP), a type of artificial intelligence, has extraordinary potential to reduce burden by automating the process of quality coding. Exposure therapy for anxiety is an ideal prototype for testing automated quality coding using NLP, given its potential for public health impact and clear theory of mechanism to guide quality measurement. Building on pilot work, this study will refine and test automated quality coding with audio data (N =1,286 patients and 11,213 sessions) from 6 existing clinical trials of exposure therapy for youth and adults, across both research and community settings. To enhance project efficiency and future utility of findings, we will leverage strong existing partnerships with 1) NIMH intramural researchers bringing expertise in the quantitative study of anxiety and deep learning methods including NLP, 2) the international Exposure Therapy Consortium (ETC), whose mission is to support centralized tools for researchers studying delivery and mechanisms of exposure therapy, and 3) a diverse set of stakeholders including patients/families, providers, agency leaders, payers, policy makers, and technology developers. Specific aims are designed to validate automated coding with NLP against human coders, test predictive utility, and incorporate stakeholder feedback to facilitate near-term usability of findings.