CNS Core: Small: Towards Timing-Predictable Autonomy in DNN-driven Embedded Systems

Machine learning techniques, particularly deep neural networks (DNNs), are enabling dramatically better autonomy in important domains, such as robotics and transportation. For instance, in automotive systems, DNNs are used to map raw pixels from on-vehicle cameras to steering control decisions. Recent end-to-end self-driving frameworks even make it possible for DNNs to learn to self-steer from limited human driving datasets. NVIDIA and Audi recently announced their plans to deliver DNN-based automated vehicles. A major challenge of safely and reliably adopting DNNs in any safety-critical embedded systems (e.g., cars) is the need to ensure timing predictability (i.e., enabling timing constraints to be analytically validated at design time), which is one of the most important tenets in the certification required for such safety-critical systems. For example, the functional correctness of an automobile hinges crucially upon temporal correctness, as the control operations depend on the processing of certain environmental sensing and computation tasks within specific time constraints. Unfortunately, it is not straightforward to achieve timing predictability in such systems, due to the resource bottlenecks that DNNs can impose. The goal of this research is to achieve timing predictability in DNN-driven autonomous embedded systems. A novel system model leveraging Heijunka, a mature production leveling approach first developed by Toyota, will be established. New DNN-aware, real-time resource allocation methods and associated analysis techniques for validating timing constraints will be developed that can be applied in DNN-driven embedded systems. Moreover, an open-source ecosystem with efficient memory management under heterogeneous hardware architectures will be implemented. The outcome of this project will pave the way to enable DNN-driven solutions to be safely and confidently adopted in many embedded domains in which timing predictability is a natural requirement. This project will also result in a pipeline of computer engineers and scientists who are skilled in the interdisciplinary nature of DNN-driven embedded systems, as well as increase awareness of real-time and autonomous system design concepts among students at all academic levels. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.