CAREER: Improving Machine Learning Education through Data-driven Support for Pipeline Design and Implementation

Machine Learning (ML) is a powerful computational method that allows computers to automatically learn patterns, or models, from data to complete complex tasks such as diagnosing diseases from medical images, translating languages, or holding human-level chatbot conversations. These technologies have incredible potential to improve lives but can also cause harm if they are not designed thoughtfully. Learning how to create ML models, and how to do so in an ethical way, is therefore an increasingly important skill across STEM disciplines. However, ML is also a challenging subject for university students to learn, and researchers have only just begun to explore effective practices for ML education. Therefore, the goal of this project is to discover new technologies that can improve student learning in ML classrooms, and to advance our understanding of how students learn ML. Specifically, the project will develop new algorithms that generate personalized feedback, hints, and examples to support students in designing effective ML models, implementing them as code, and reflecting on the ethical implications of their design choices. These help features will be automated, allowing them to scale to classrooms of any size, supporting students even when the instructor is unavailable. This project will evaluate and improve these technologies through studies in three different classrooms over five years, potentially improving outcomes for over 1,500 students. Additionally, by analyzing log data, surveys, interviews, and student outcomes, this project will advance researchers' understanding of how students learn ML and inform the design of interventions to support them. The project will have further impact by developing and distributing an "ML for Social Good" curriculum and by disseminating the technologies it develops through multiple channels, including professional development workshops for 16 instructors. This project will discover methods for supporting students in learning to design and implement open-ended ML pipelines. It will do so through novel, data-driven algorithms that generate personalized feedback, hints, and examples to support students. These algorithms will use trace data from prior students to generate support for future students, effectively using advances in ML to support ML education. While prior data-driven support is primarily used in early computer science courses to support novices, the proposed research will advance these techniques to support more advanced programmers working on complex and open-ended ML projects. The three thrusts of this proposal will answer fundamental questions in ML education research: Thrust 1 will investigate the value of an explicit ML design process that separates planning from implementation and will explore how a digital interface can scaffold this process and facilitate instructor feedback. Thrust 2 will investigate how to design automated hints and feedback to effectively support ML pipeline design and the comparative affordances of immediate algorithmic help vs delayed instructor help. Thrust 3 will explore how students use, integrate, and learn from programming examples when implementing ML pipeline code and will develop methods to personalize these examples using student data. This project will evaluate the impact of automated support in laboratory and classroom studies, reaching hundreds of students each semester across multiple courses and instructors. This research will make contributions to the fields of computing and ML education research through systematic analysis of students' behavior and artifacts to further our understanding of the processes by which learners acquire and demonstrate knowledge when constructing ML pipelines. The project will also advance techniques for automated programming and design feedback, which will build a foundation for future research supporting other advanced computing courses. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.