SHF: Small: Software and Hardware Support for Robust Deep Learning

Deep neural networks have made unprecedented strides in recent years, making them a ubiquitous component of any modern computing system. Deep neural networks and their hardware will be commonplace in future in various systems ranging from simple tasks to safety and security-critical systems. Therefore, it is imperative to have specialized support to improve the robustness of deep neural network models so that users and developers can trust the predictions provided by those models. Towards that end, this project presents novel algorithms for robust deep neural networks. The project aims at ensuring robust operations of deep neural networks. The project, called GuardiaNN, has two components - dynamic monitoring schemes and testing schemes. The goal of dynamic monitoring is to determine whether a deep neural network model can misbehave during inference at a production setting. Two novel monitoring algorithms are considered in this project - neuron pattern and model-based monitoring. Neuron pattern-based schemes work by first extracting simple neuron activation patterns and then, later at runtime, checking whether those patterns are satisfied or not. Model-based schemes work by first determining a minimal set of data points to monitor and then, using an advanced model to detect if those data points show any anomalous pattern or not. The goal of testing schemes is to determine how the model might behave in corner case scenarios. Testing allows users to explore the behavior of the model without understanding the internal details of the model. Together the monitoring and testing approaches will provide trust in deep neural network operations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.