SHF: Medium: A Neursoymbolic Framework for High-level Synthesis of Multi-Task Learning (NeuHLS)

The growing demand for smart and autonomous systems has driven a surge in the deployment of edge devices. However, the limited computational resources and energy constraints of these devices pose significant challenges for deploying complex deep neural networks (DNNs). Optimizing DNNs for edge devices is crucial to unlock their full potential and enable a wider range of innovative applications. This project’s novelties lie in developing a new generation of tools that can automatically generate hardware accelerators for edge devices while satisfying latency and hardware platform constraints. This project’s impact is to enable high-performance DNN models with high accuracy and fast response to be synthesized in constrained hardware such as Virtual Reality (VR)/Augmented Reality (AR) or assistive robotics will positively change social perception and confidence towards using these future ubiquitous systems. Our approach integrates multi-task learning, neurosymbolic Artificial Intelligence (AI), and high-level synthesis to create accelerators that meet strict latency and hardware platform constraints. In particular, this project introduces NeuHLS, a neurosymbolic approach for merging, compressing, and synthesizing DNNs. NeuHLS’s primary objective is to develop a flexible and efficient framework that balances accuracy, hardware utilization, and latency. In addition, the synthesized hardware must maximize the number of DNN weights implemented using software tunable parameters to allow for flexible fine-tuning at runtime. The proposed NeuHLS toolchain consists of three phases. The first one aims to merge a set of single-task DNNs into one multi-task DNN by sharing representations from different single-task DNNs, hence reducing the model size. Next, the toolchain exploits recent advances in symbolic knowledge distillation to compress the multi-task DNN into a neurosymbolic model, which can then be processed by novel neurosymbolic high-level synthesis techniques that optimize the deployment while balancing accuracy, hardware utilization, and latency. The team of researchers will evaluate the toolchain using existing benchmarks and real-world application deployment in the various domains of autonomous systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.