Collaborative Research: Developing and Testing a Customizable Intelligent Tutoring System for Undergraduate Quantitative Methods Education

This project aims to serve the national interest by harnessing the power of artificial intelligence (AI) to develop stronger quantitative research skills among undergraduate STEM students. Quantitative research methods are among the most challenging courses undergraduate students face, yet mastery of these skills is essential for success in many science and professional fields. Despite their importance, students often struggle with the material, and instructors have limited tools for providing personalized support at scale. This Level 1 Engaged Student Learning project will develop and test a customizable intelligent tutoring system (ITS), a computer-based application that uses artificial intelligence (AI) to deliver personalized feedback and support as students work through quantitative methods and statistical content in a required research methods course for psychology majors. Instructors will be able to customize the system for their own courses without specialized technical skills. A study with approximately 1,000 undergraduate psychology students will test how well the system supports learning and persistence in quantitative coursework and in STEM. The project will co-design the Intelligent Tutoring System with experienced instructors teaching undergraduate quantitative methods, ensuring the system addresses real classroom needs and constraints. The system will provide students with lessons on core course content, including scientific writing and statistical analysis, standardized across course sections with the ability to be customized by individual instructors through a user-friendly interface that requires no technical expertise. Generative AI technologies will enable lesson dialogue and writing analysis tailored to individual student needs. The Intelligent Tutoring System will be piloted in participating classrooms to examine usability, learning outcomes, and factors that influence effectiveness in a naturalistic setting. Longitudinal follow-up data will assess the lasting effects of Intelligent Tutoring System use on students' science self-efficacy and STEM persistence. The project will generate knowledge about how intelligent tutoring systems can be designed and customized by educators, what factors influence their effectiveness, and how personalized AI-based instruction can improve undergraduate quantitative methods education. The NSF IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.