SLES: Whitebox Testing, Debugging, and Repairing for Multi-module Autonomous Vehicles in Near-Collision Traffic Scenarios

The advances in artificial intelligence and machine learning have empowered the development and adoption of autonomous vehicles, including self-driving cars and delivery drones. However, the increasing number of incidents involving autonomous vehicles has raised public concerns about their safety and reliability. Ensuring end-to-end safety of such systems is critical but challenging given the sophisticated multi-module systems operating in these vehicles and the enormous number of possible traffic scenarios, especially complex and previously unseen scenarios. Though many testing and verification approaches have been proposed, they are mainly designed for a single vehicle in simple scenarios, which limits their applicability to modern multiple-module systems in which multiple models and conventional algorithms are used in tandem for perception, prediction, planning, and control. This project seeks to reason about the inherent interaction among multiple modules in an autonomous vehicle to systematically identify, debug, and repair unsafe behavior in realistic and diverse scenarios. It will provide empirical assurance of and boost public confidence in the end-to-end safety of these vehicles. Techniques developed in this project will be open-sourced and will be broadly available for building safe robotic systems in various sectors. The project integrates research and education through curriculum development, student advising, and K-12 outreach activities with a focus on recruiting and mentoring students from underrepresented minority groups. This project will develop principled algorithms and practical tools that systematically discover unsafe behavior in a system via a deep exploration of realistic and diverse traffic scenarios and repair the system to enhance end-to-end safety. The key contributions include (1) a method for automated test-scenario construction that decouples high-level semantics and low-level details through a novel Domain Specific Language-based synthesis algorithm, (2) a search-based testing method that efficiently explores the enormous space of possible scenarios and identifies collision-inducing scenarios through a layered abstraction of multi-module autonomous systems and hierarchical optimization, and (3) a new adaptive debugging and repair technique that strategically diagnoses and fixes different kinds of safety bugs in different modules at different levels of granularity. The safety enhancement achieved by the developed framework will be rigorously quantified and validated both in simulation and in physical vehicles. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.