Exploring the Value of LLMs in Teaching Introductory Programming

This project aims to serve the national interest by redesigning an introductory computer science course (CS1) to teach students how to write computer code with the aid of an artificial intelligence (AI) assistant. The rise of Large Language Model (LLM) tools such as ChatGPT and GitHub Copilot has created new challenges and opportunities for education. In computer science education, both enthusiasm and concern exist regarding how to best teach students to write code in the presence of these AI assistants. The proposed course called CS1-LLM intends to teach students skills different from the skills taught in traditional CS1 courses. In the proposed CS1-LLM course, less emphasis will be placed on syntax and writing code from scratch, but more emphasis and attention will be given to code reading, code testing, problem decomposition, and writing software in meaningful contexts. This proposed project will be built upon early results from a pilot course offered at UCSD in Fall 2023. This proposal seeks to build on the initial findings by: 1) redesigning the course to improve student success across a wide range of metrics, 2) studying how students will learn programming with the aid of an LLM, 3) comparing how well students can write software a year after completing a traditional CS1 course vs a CS1-LLM course, 4) examining how the inclusion of LLMs in a CS1 course will impact learning outcomes of the students from underrepresented groups, and 5) engaging instructors worldwide in this study to understand their perspectives of adopting such a course. The NSF IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students. Through its Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.