Using Large Language Models to Support Authoring Intelligent Tutoring Systems

There is a significant amount of scientific research that shows that intelligent tutoring systems (ITS) can help students learn better than through other forms of instruction and software. These systems give frequent, fine-grained guidance when students practice solving complex problems. ITS have proven to be useful in many domains, for example, STEM, business, and language learning. The primary goal of this project is to investigate whether and how Large Language Models (LLMs) can help instructors author ITS effectively and efficiently – a process that needs to be executed each time an ITS is created for a new content area. The project will provide insight into how LLMs can best assist in building ITS and in the efficiency gains that result. The project has the potential to help spread ITS widely into educational practice and thereby help many students learn better, from elementary school to college-level and even graduate school. To this end, the project will integrate existing LLMs with an existing set of authoring tools: the Cognitive Tutoring Authoring Tools (CTAT), which have long supported the efficient development of ITS, in two paradigms offering different capabilities. The project will assess whether and how LLMs can help with two time-consuming authoring tasks: generating a wide range of practice problems and developing rule-based cognitive models that capture the problem-solving knowledge that the ITS aims to help students learn. While assistance from an LLM is added, instructors remain in charge and contribute their specialized pedagogical content knowledge and skill in instructional design. The project will develop methods for authors to ensure that the generated content will be accessible and appealing to all students. The project will conduct evaluation studies to measure both the quality of the tutoring systems authored with LLM assistance and the gains in authoring efficiency that may result. This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.