SHF: Toward Continuously Evolving Software Engineering Agents

Bugs in modern software systems can affect millions of people worldwide and cost billions of dollars in financial losses. To eliminate software bugs, state-of-the-art solutions typically leverage LLM (Large Language Model) agents, which can invoke external tools and autonomously produce a trajectory of actions to solve end-to-end software tasks (such as fixing reported issues). While achieving impressive scores on popular benchmarks, such software agents are still limited due to the suboptimal training data, ad-hoc agent design, and uninformed trajectory exploration. This project aims to fundamentally transform modern software agents by developing a series of novel strategies that enable LLMs to continuously and autonomously evolve their own training data synthesis, agent scaffold design, and trajectory exploration. The project's broader significance and importance are: 1) it could result in a series of LLMs and agents for autonomously solving real-world software issues and beyond, benefiting developers worldwide; 2) it could substantially advance state-of-the-art LLMs for coding and reasoning, which could further boost more downstream applications; 3) the research results could also be integrated into the software engineering curriculum for educating the next generation. More specifically, the project covers the following three research thrusts: 1) Continuously Evolving Model Training, which trains the underlying LLMs for both synthetic issue generation and issue resolution. The joint training paradigm will guide LLMs to learn to generate the most valuable issues to further improve their own issue-resolution capabilities in a self-play loop. 2) Continuously Evolving Agent Design, which further builds live software agents that will reason about and improve their own scaffold implementation at runtime on the fly for maximal bug resolution. 3) Continuously Evolving Trajectory Exploration, which further enables the agents to build on their earlier trajectories and progressively improve their performance. All three thrusts together also form a holistic ecosystem in which continuous, autonomous evolution at different levels can mutually reinforce one another for even more powerful software engineering support. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.