SHF: CORE: Small: Exploring the Uses of Large Language Models to Assist Program Analysis

Computer programs today are essential to everything from smartphones and vehicles to national defense. However, the software that runs these systems is often large and complex, making it difficult to detect hidden problems before they cause real-world harm. Existing tools used to find flaws, such as bugs or vulnerabilities, can either miss important issues, report many false alarms, or take too long to run. This project will investigate how artificial intelligence, specifically Large Language Models (LLMs), which are systems trained to understand and generate text (including code), can assist in the analysis of software. Instead of using LLMs by themselves, this project explores how they can support existing analysis methods and tools, improving their accuracy and speed. The successful completion of the proposed activity will lead to changes in the way programs are analyzed to find bugs and vulnerabilities. The project hypothesizes that LLMs can be used as a complementary approach to conventional program analysis by combining the strengths of both to address key challenges in analyzing large and complex software. The research will begin by identifying the limitations of current program analysis techniques, such as the tradeoffs they must make between precision and scalability. It will then explore strategies for using LLMs in supportive roles to target these limitations and assist with particularly difficult aspects of program analysis. The design space for integrating LLMs with existing tools is broad and includes possibilities such as bug-type-specific workflows, autonomous agents, and mechanisms for verifying the correctness of LLM-generated results. The project will evaluate these designs in real-world applications, including tools for vulnerability discovery and enhanced operating system testing. The expected outcome is a set of more effective and practical analysis tools that advance both software security and the understanding of how to combine artificial intelligence with program analysis methods. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.