Collaborative Research: CSR: Small: Expediting Continual Online Learning on Edge Platforms through Software-Hardware Co-designs

Deep neural networks (DNNs) have gained significant popularity in emerging application domains such as robot-assisted eldercare, mobile diagnosis, and wildlife surveillance. These applications commonly (i) employ continual online learning that fine-tunes the DNN model based using streaming-in training data to serve overtime inference requests and (ii) deploy the DNN models on energy-constrained edge devices. As such, both model adaptiveness and device energy efficiency are critical for user satisfaction. This research uncovers redundancy in fine-tuning and enhances the computation efficiency to achieve practical, efficient, and adaptive continual online learning on edge devices. This project's educational and outreach components include (i) curriculum and course project expansion on deep learning and edge computing. (ii) Engaging undergraduate students in research activities through senior course projects and outreach programs at PIs’ institute. (iii) Increasing the participation and visibility of female and minority students in computer science and engineering. This research aims to simultaneously achieve adaptiveness and energy efficiency for continual online learning on edge devices. (i) It develops an attention-guided smart layer freezing to reduce computation costs by automatically and dynamically freezing converged layers. (ii) It designs an efficient in-situation learning framework for edge devices. The framework selectively delays and merges fine-tuning iterations to reduce the fine-tuning frequency and handles scenario changes. (iii) It designs hardware-support memorization to reduce the amount of fine-tuning computation and memory accesses. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.