Reasoning with Embodied AI Labs in STEM Education

Hands-on laboratory experiences help students connect scientific ideas to the observable world and develop reasoning and inquiry skills. Yet many middle and high schools lack the resources, time, and scaffolding needed for open-ended investigation, so laboratory activities often become scripted procedures with limited feedback or exploration. The project will address this challenge by developing algorithmic models and software applications powered by embodied artificial intelligence to bring the advantages of virtual labs into physical learning environments. Students will conduct experiments in physical labs while receiving timely explanations, prompts, and visualizations tailored to their actions and experimental conditions. These supports will help learners interpret phenomena that are difficult to see directly, including thermal radiation, force trajectories, and ecosystem change. By making inquiry-based science learning more accessible and responsive to varied learning needs, the project will strengthen scientific literacy and preparation for future study and work in science and technology. The project will also support teachers, broaden participation in meaningful science inquiry, and produce open resources for researchers, educators, and learning-technology developers. Technically, the project will develop browser-based applications and embodied artificial intelligence models for laboratories in physical and biological sciences. The system will integrate computer vision, cyber-physical sensing, multimodal generative artificial intelligence, and vision-language-action architectures to interpret student actions and experimental conditions in real time and provide personalized feedback, scaffolds, and augmented reality visualizations. Guided by the learning theory of concreteness fading, these supports will connect concrete manipulation of lab materials with progressively more abstract scientific representations. The research team will use iterative design-based research to create, pilot, and refine the applications and curriculum with teachers and students in authentic middle and high school classrooms. The evaluation will examine effects on students’ conceptual understanding, scientific inquiry practices, and self-efficacy of science learning. The project will also investigate technical challenges in real-time multimodal perception, planning, feedback, and rendering within dynamic and sensor-rich environments. Expected contributions include an openly released multi-agent software framework, specialized models, source code, project-generated datasets, and empirical evidence about how embodied artificial intelligence can support learning. These outcomes will advance research at the intersection of artificial intelligence, learning sciences, and science education while establishing a foundation for responsible and scalable intelligent laboratory experiences. This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.