Artificial Intelligence-Assisted Real-Time Illustrative Figure Generation for Engineering Classroom Engagement

This project aims to serve the national interest by improving how engineering instructors explain difficult concepts to undergraduate students. Clear diagrams help students grasp complex ideas, yet good illustrations take time and expertise to prepare, and are rarely available when a student raises an unexpected question. This Level 1 Engaged Student Learning project plans to develop and study an open, artificial-intelligence-powered tool that will generate a custom illustrative figure in under a minute during class, with the instructor reviewing and approving every figure before students see it, keeping the technology a teaching aid rather than a replacement. By making a well-documented but underused practice (simple visual explanation) affordable and broadly accessible, including at teaching-focused institutions and community colleges that lack instructional-design support, the project has significant potential to reduce the cognitive overload associated with engineering attrition, with importance for student persistence and success. The project’s goals will be to develop, refine, and rigorously assess an open-source platform that will generate context-sensitive illustrative figures during live instruction. The scope will cover the full generation pipeline together with classroom validation across several engineering disciplines at a single research university. The platform will convert a spoken student question into a classroom-ready figure through an automated workflow. It will a) transcribe the question on the instructor’s own computer, b) use large language models to analyze the concept, c) compose an optimized image-generation prompt from that analysis, d) produce the figure with an open diffusion model fine-tuned on educational illustration datasets, and e) return an automated accuracy check, with instructor approval required before display. The design will draw on established learning-science theory, including Cognitive Load Theory and the Cognitive Theory of Multimedia Learning. The project plans to investigate the conditions under which these real-time visuals improve instruction through iterative pilot testing, multi-course deployment, instructor interviews, student surveys, classroom observations, and analysis of system logs. Assessment will include generation speed and reliability, expert and instructor ratings of pedagogical quality and usability, and early indicators of student understanding, and the resulting platform, instruments, and de-identified data will be shared openly with the field. The NSF IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.