CRII: SHF: RUI: Enhancing LLM Debugging Capabilities: A Study on AI-Driven Software Error Detection and Correction

Software has been an essential part of daily life, embedded in the tools and applications people use every day. When software fails - for example, when a mobile app crashes or a medical system displays incorrect information - it can lead to frustration, wasted time, or even safety risks. These issues are often caused by mistakes in the code, commonly referred to as "bugs". Fixing bugs, known as "debugging", is often time-consuming and requires considerable problem-solving skills. As software systems grow in complexity, there is an increasing need for tools that help developers identify and fix bugs more efficiently and accurately. This project will explore how recent advances in artificial intelligence - particularly large language models (LLMs) - can support and enhance the debugging process. The work will lead to the development of smarter tools that reflect how people approach problem-solving, making it easier for developers to correct errors. These improvements could lead to more reliable software across fields such as healthcare, transportation, education, and beyond. The project will focus on three main research activities. (1) Developing a comprehensive benchmark to evaluate the performance of LLMs across a range of software bug types, including logic, functional, and performance-related errors. This benchmark will fill a critical gap by providing a structured framework for systematically assessing LLM capabilities in debugging tasks. (2) The project will incorporate developer-like problem-solving strategies, such as understanding problem context and analyzing failed test cases, into the design of LLM-driven tools, enhancing their ability to detect and explain bugs. (3) This project will streamline the debugging workflow by leveraging human-AI collaboration to deliver more reliable and actionable solutions. Through the integration of feedback mechanisms, reinforcement learning, and iterative refinement, it will enhance the ability of LLMs to generate fixes that are more accurate, efficient, and practical, thereby reducing the need for manual correction. These innovations are expected to significantly enhance developer productivity and improve overall software quality. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.