Empowering Educators to Self-Author Virtual Humans for Interpersonal Skills Training

Training the next generation of U.S. healthcare workers requires teaching real-world communication skills that are difficult to practice in traditional classroom settings. This project develops a virtual simulation system for clinical communication skills that can be easily customized by instructors and allow students unlimited, realistic practice talking to patients. Researchers use artificial intelligence (AI) to power virtual patients with unscripted and realistic verbal and non-verbal behaviors. The simulations take place in a virtual reality (VR) environment that further enhances students’ feelings of being with a real patient and needing to think on their feet. The system allows educators who do not have technical expertise in AI or VR to create patient cases, ensuring the simulations reflect the competencies and skills that emerging healthcare professionals need to succeed. This research advances learning technologies by developing teaching tools that are flexible to the needs of instructors and students, resulting in a new, more effective approach to preparing future healthcare professionals that can be scaled nationwide. This project places the development of AI-powered virtual patients directly in the hands of clinical health educators, advancing both the technology and the evidence base for its effective use. Grounded in design-based research, situated learning theory, and affective computing, this project aims to: (1) develop and test an educator-controlled AI patient designer for VR simulations, (2) identify best practices to make virtual interactions feel real to maximize learning, and (3) develop performance assessments that go beyond measuring content retention. Across these three interconnected aims, the research team builds and refines an authoring environment that empowers instructors to create high-fidelity virtual patient simulations, examines student behavioral responses using eye tracking, gesture and speech, leveraging clinical faculty expertise throughout the research process. A classroom implementation study compares the system against standard instructional practice and assesses learning transfer to real-world clinical performance. Quantitative and qualitative data from students and faculty provide a rich, mixed-methods evidence base for effective clinical education with emerging technologies. This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.