Collaborative Research: SHF: Small: Designing and Optimizing Tiny Vector Symbolic Architectures for Ultra-Efficient Inference on Tiny Devices

Despite the advances in deep neural networks (DNNs) and edge computing, there exist substantial challenges to enabling end-to-end DNN inference on a full spectrum of edge devices, such as tiny wearables and low-cost Internet-of-things (IoT) devices. This problem has spurred the recent studies of brain-inspired vector symbolic representation (VSA) classifiers as an alternative framework for ubiquitous on-device inference. At a high level, VSA classifiers mimic the brain cognition process by representing each object as a vector (typically in a very high-dimensional space). While VSA classifiers offer advantages over DNNs in terms of inference efficiency due to parallel processing, the hyper-dimensionality in their design can still easily result in a prohibitively large VSA model size beyond the limit of many tiny devices with stringent resource constraints. If successful, this project will make it possible for more everyday devices to run advanced artificial intelligence (AI) on their own, without needing to send data to remote servers. This could improve privacy, save energy, and open the door to smarter wearables, medical devices, and home gadgets. Finally, the project will bring the latest discoveries into college courses to help train the next generation of engineers and computer scientists. To address the hyper-dimensionality challenge, this project moves away from hypervector-oriented VSA and proposes TinyVSA, which uses much smaller, compact vector representations. Specifically, this project focuses on three key directions: first, redesigning TinyVSA’s vectors to improve accuracy while the VSA dimensionality by orders of magnitude; second, making TinyVSA run continuously and efficiently on tiny, low-power chips; and third, developing an efficient, hardware-aware method to automatically find the best TinyVSA architecture for target devices. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.