Collaborative Research: EAGER: Automating CI Configuration Troubleshooting with Bayesian Group Testing

Configuration troubleshooting in large-scale cyberinfrastructure (CI) software systems is a complex and costly task due to numerous configurable parameters. Existing methods like log mining and machine learning analysis face challenges in such environments. To address this, we present BGT4AutoCI (Automating CI Configuration Troubleshooting with Bayesian Group Testing), a groundbreaking solution that leverages Bayesian Group Testing, ensuring accurate results even with minimal prior knowledge and testing errors. Experienced CI operators can expedite the process with approximated prior knowledge. This research aims to revolutionize CI configuration troubleshooting, introducing a highly precise and efficient approach that will optimize the utilization of current and future large-scale CI systems. The primary focus of this research is to address critical challenges in automated configuration troubleshooting within large-scale CI systems. The proposed three-fold approach encompasses: (1) Formulating Bayesian Group Testing for CI configuration troubleshooting, which employs lattice models to accurately identify risks at the individual configuration parameter level, taking uncertainty into account during troubleshooting. (2) A multinomial paradigm for Bayesian Group Testing, which introduces multinomial responses to simultaneously observe multiple aspects of CI systems, enabling efficient test selection algorithms for jointly testing configuration parameters that impact various aspects of CIs. (3) Automated configuration troubleshooting, which involves the designs of several key components to establish BGT4AutoCI as an automated configuration troubleshooting framework that minimizes the need for human intervention. The outcomes of this project hold the potential to significantly enhance the efficiency and accuracy of CI configuration troubleshooting, benefiting current and future large-scale CI systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.