REU Site: Trust and Reproducibility of Intelligent Computation

This Research Experience for Undergraduates Site addresses the growing reality that modern society is increasingly dependent on complex software components to work as expected (trustworthy) and produce consistent results (reproducible). Trustworthiness and reproducibility govern adoption and acceptance of the results produced by software across a range of applications including medical diagnostics, facial recognition, traffic analysis, network security, and chemical reaction control. Computing educators are obliged to instill in the next generation of scientists and engineers -- today’s undergraduates -- an understanding of principled methods that enhance software system reliability, trustworthiness, and reproducibility. A typical undergraduate student is not sufficiently exposed to these concerns nor the aforesaid supportive methods, and yet they will be the ones building future intelligent systems. Thus, this REU site will address these issues, as well as emerging dangers such as introducing bias into AI-based applications or leaking personal data demand instruction in ethical considerations of software systems, another aspect of trustworthiness. Student participants will learn the state-of-the-art methods typically used in trustworthy and reproducible science and engineering. Weekly training and activity sessions will bring the entire cohort together for pre-packaged exercises, e.g., using Jupyter notebooks, software version control, automated defect detection, automatic performance analysis and optimization, and data analysis. These activities are deployed on one-of-a-kind, translational research platforms operated by the University of Utah, namely the NSF-funded Cloudlab and the POWDER project. Additional activities through the university’s Office of Undergraduate Research and the Utah Center for Inclusive Computing allow the researchers to offer workshops on research best practices, ethics, and inclusion. With a focus on applications that incorporate machine learning, achieve high efficiency, and the systems that support the applications, each undergraduate participant will work with a faculty mentor and their research group to complete a research project, producing both a written research report and a well-packaged artifact that, together, enable another person to understand the research, repeat the experiments, and reproduce the results. Students are selected to the program with the dual goals of broadening participation in computing and offering research experiences to students with severely limited opportunities at their home institutions. The program is assessed at the beginning and end, with an evaluation of how students applied their new understandings to their research projects, with follow-ups conducted as students apply to graduate school. The project plans to share these assessments and the curricular material with other institutions to propagate education in trustworthy and reproducible software to galvanize the next generation of scientists and engineers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.