Collaborative Research: Supporting Students' Modeling Practice Through a Large Language Model-Powered Pedagogical Agent in an Augmented Reality Learning Environment

Modeling is a key scientific practice emphasized in the Next Generation Science Standards (NGSS), but students often need significant support to engage in the practice meaningfully. This project aims to develop an augmented reality (AR) learning environment integrated with a large language model (LLM)-powered pedagogical agent to guide middle school students' modeling practice. It will bring together computer scientists, human-computer interaction researchers, science education researchers, and K-12 teachers to co-design the learning environment. The AR environment will allow students to interact with simulated objects and phenomena as they develop their models. The LLM-powered agent will provide timely assessments of students' diagrammatic models, offer personalized feedback, and share insights with teachers to inform instruction. This project has the potential to transform the way students engage in modeling practice in K-12 science classrooms. Over three years, it will directly impact approximately 10 middle school teachers and 400 middle school students. The project is guided by three core objectives: 1) to build AR simulations for modeling tasks, 2) to develop an on-device multitask LLM to assess diagrammatic models, and 3) to investigate student modeling practice within the AR-LLM learning environment. A key innovation of the project is the closed-loop feedback system: students first receive suggestions for model revisions from the LLM and revisit AR components, and the LLM agent then provides feedback on students' performance to teachers to inform instructional decision-making. The project will investigate students' perceptions of their modeling experiences and how the AR elements and LLM-generated feedback support their model development respectively. Primary data sources will include classroom video recordings of student consensus model building, screen recordings of individual modeling activities, and semi-structured focus student interviews. A multiple-case study approach will be used to investigate student learning. Research products will be widely shared with practitioners, teacher educators, and researchers through publications, conference presentations, teacher workshops, and publicly accessible teacher resources. This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.