HCC: Small: Advancing Strengths-Based Approaches to Building Personalized, Safe Large Language Model-based Agents through Developing AI-Based Coaching Assistants for Job Seekers

This project aims to develop methods to create individually tailored large language models (LLMs) in the context of systems for strengths-based coaching of job seekers. People with autism have a wide range of abilities and needs around the interactions needed to conduct a job search. Strengths-based approaches to job coaching, which identify and apply each person's unique strengths in preferred work environments, show promise in helping individuals secure competitive employment, but the high cost and labor involved make strengths-based coaching inaccessible to many young adults. Currently, individuals turn to tools like ChatGPT to help with cover letters, interview preparation, and other job-seeking tasks. However, these tools often produce generic responses that fail to reflect the unique strengths of job seekers and raise ethical concerns about handling sensitive, diagnosis-related information and the risk of over-reliance on the LLMs. Through developing new ways to build personalized LLM-based agents that discover and incorporate people's strengths and needs, this project will both advance the utility and safety of LLMs and increase the employment projects for individuals with autism and others, benefiting society. To address these challenges, the project will follow a three-part approach. First, the project team will analyze chatbot dialogue data and neurodivergent user profiles from the LLM job coach feature currently deployed on a neurodiversity-focused employment platform that connects neurodivergent job seekers, including autistic individuals, with employers. Insights from this analysis, along with input from individuals and professional job coaches, will guide the development of job coaching guidelines, identify opportunities for strengths-based support, and uncover ethical risks and safeguards to inform the design of a strengths-based job coaching LLM model. Second, the project team will create a two-part system based on the identified design guidelines: a fine-tuned LLM for job coaching and a control model that uses individual strengths to generate personalized, context-specific responses. This design addresses the limitations of generic, one-size-fits-all outputs. Third, the project will build a novel job coaching system leveraging these advances and test how effectively the developed strengths-based job coaching LLM can support job seekers with autism and others. Through co-design workshops and lab-based evaluations, the project will assess the system's ability to improve people's confidence, self-esteem, and perceived usefulness of the coaching process, while identifying areas where the model may need refinement to ensure ethical and practical implementation at scale. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.