SLES:Orchestrating Model, System, and Hardware for Safe Learning in Autonomous Vehicles

Machine Learning (ML) has emerged as a transformative force in advancing numerous high-stake domains, particularly in the realm of autonomous driving. With the rise of computationally powerful ML techniques, autonomous vehicles (AVs) can understand their environment with high precision of perception, make real-time decisions, and operate reliably without human intervention. As AVs are safety-critical, the end-to-end safety of its learning system is essential. However, our research reveals that potential unsafety may stem from multiple aspects, including the model, the hardware, or the system. First, ML models trained predominantly on common driving scenarios may extrapolate inappropriately when encountering unique road obstructions or rare weather conditions, leading to incorrect driving decisions or unsafe vehicle control. Moreover, the hardware platforms executing the ML models are imperfect and suffer from various types of faults and errors. Last but not least, when multiple ML models are executed concurrently, the real-time operating system (RTOS) of AVs may not deliver the decisions in time, leading to catastrophic consequences. Developing a safe learning-enable system for AVs requires orchestration of the model, the hardware, and the system. Bringing together experts from machine learning, hardware fault tolerance, and autonomous driving, this project focuses on cross-layer optimizations to achieve end-to-end safety. Key innovations include developing rational ML models to produce accurate predictions based on valid rationales, integrating hardware reliability into ML model design to tolerate runtime faults, and designing a novel RTOS scheduler that ensures time predictability while considering model and hardware reliability. The project will implement these advancements on real autonomous driving platforms to validate their effectiveness. The success of this initiative will significantly enhance AV safety, promoting safer, cleaner, and more efficient transportation. Additionally, the project will advance education and workforce development in AI and autonomous driving, with a commitment to diversity and inclusion in STEM fields. Results and software will be disseminated through open-source platforms, educational programs, tutorials and workshops, fostering broader impacts on technology and society. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.