CSR: Small: Predictable Multi-Tenant DNN Inference for Autonomous Driving

Deep Neural Network (DNN) technology has achieved significant success in autonomous driving systems, particularly in environment sensing and perception tasks. However, ensuring the timing predictability of DNN decisions during operation of autonomous vehicles (AV) is crucial for safety. Substantial time variations persist in most DNN models within AV systems, and this project’s novelties are systematically controlling such variations for autonomous vehicles secure operation. The project's broader significance is enhancing the reliability and safety of autonomous vehicle perception systems, ultimately reducing accidents and improving road safety. This project consists of three research thrusts: (1) understanding the challenges of timing predictability in DNN inference for autonomous vehicles; (2) designing a framework for predictable DNN inference in multi-sensor and multi-task AV perception; and (3) integrating this framework into Autoware, a real AV pipeline. The project develops a configurable profiling framework to comprehensively understand the root causes of variability in the DNN inference pipeline. This framework allows fine-grained profiling of time variation issues, including data variability (sensor, weather, and traffic scenarios), model variability, and runtime system variability (communication middleware, operating system, and hardware architecture). To mitigate DNN inference time variations and ensure predictability, this project addresses single DNN inference variations through feature maps caching and fusion techniques. Additionally, multi-tenant DNN inference is optimized through co-scheduling across the application, middleware, operating system, and architectural layers. The team integrates the multi-tenant co-scheduling framework into Autoware, creating a lightweight message-wise timeline checkpoint with a feedback-based co-scheduler. Comprehensive evaluations are conducted using open AV datasets, an indoor connected and autonomous testbed (ICAT), and a 2018 Lincoln MKZ-based level-4 AV equipped with Autoware at the University of Delaware. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.