Research Infrastructure: Mid-scale RI-1 (M1:DA): A Generative AI Learning Agent Infrastructure for STEM Education Research

AI tools that can tutor students, provide personalized feedback, and adapt instruction to individual needs are rapidly transforming STEM education, yet many education researchers may not have learned software engineering skills to build such AI agents, forcing research teams to acquire new skills and slowing the pace of research. This project aims to design a research infrastructure that allows STEM education researchers to design, implement, test, and analyze learning agents without advanced programming expertise. Using this infrastructure, education researchers can describe learning objectives and teaching strategies in plain language. The system will then generate the working software (for the intended learning agent) capable of interacting with students, tracking what each student learns, and continuously adapting to individual learning needs. The infrastructure also conducts automated testing, which enables researchers to check whether the agent behaves as intended, and the infrastructure collects data with granular logging of student-agent interactions. To address the challenges in developing these capabilities, the research team will take a generative AI approach and exploit the remarkable code generation ability of large language models (LLMs). The research team will implement a system prototype of the designed infrastructure and demonstrate its core capabilities. If successful, the infrastructure would enable education researchers to build a variety of STEM learning agents in days instead of months. It democratizes AI by removing technical barriers, allowing more education researchers to conduct quality research. The research team will create a community-supported web portal where students and researchers nationwide can access resources, share their infrastructure usage experiences, and form working groups. This project addresses four infrastructure gaps limiting STEM education research on generative AI learning agents. The first is the domain-specific implementation barrier: education researchers possess disciplinary and pedagogical expertise but may have to rely on technical collaborators with software engineering skills, introducing a knowledge-transfer bottleneck. The second is the complexity of agent validation, which must simultaneously assess pedagogical effectiveness and technical correctness. The third gap is the inadequacy of traditional data collection methods for capturing multidimensional student-agent interaction patterns. The fourth is the difficulty of grounding LLM outputs in verified domain knowledge to reduce factually incorrect responses, a process requiring technical expertise beyond most researchers’ training. The proposed infrastructure addresses the four gaps through five integrated phases. In phase one, researchers upload the teaching materials, and the infrastructure generates a discipline-specific knowledge graph, capturing concepts, skills, and their relationships. In phase two, the infrastructure identifies integration points where AI agents can become part of a student’s learning process. The corresponding student-agent interaction designs will then be suggested by the infrastructure based on well-known interaction patterns in the field of learning science and pedagogical best practices. In phase three, the infrastructure automatically generates working agent software using a coding assistant purpose-built for STEM education and a retrieval-augmented generation architecture that grounds agent responses in the knowledge graph and student performance records, directly mitigating LLM hallucination. In phase four, the infrastructure generates test cases, identifies incorrect outcomes, and refines agent behavior. In phase five, the infrastructure’s continuous logging system captures student-agent interactions. The collected research data also includes student profiles, agent performance, and student learning patterns. The infrastructure design will be validated across several STEM fields through controlled studies. All infrastructure components will be released as open-source software, with knowledge graphs, prompt templates, and tools shared via a web portal, and workshops will be organized to foster an active infrastructure user community. This project is ultimately likely to strengthen STEM workforce development through resulting in widescale AI-supported high-quality approaches to STEM learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.