CAREER: Differentiable Network-Accelerator Co-Search Towards Ubiquitous On-Device Intelligence and Green AI

There exists a vast and increasing gap between the prohibitive complexity of powerful Deep Learning (DL) algorithms and the constrained resources available for implementing them. It has been recently recognized that jointly designing the DL algorithms and their hardware accelerators is very promising in closing this vast gap. However, existing works have just begun to scratch the surface of its full potential. This project aims to foster a systematic breakthrough in developing DL accelerators and their achievable acceleration efficiency by jointly searching (co-search) for DL algorithms and their accelerators. The overarching goal of this project is to develop, implement, and experimentally validate a new paradigm of designing deep DL accelerators for enabling: (1) orders of magnitude faster development speed; (2) much improved hardware efficiency, and (3) unprecedented flexibility to control the trade-off between hardware efficiency and task performance, by holistically fostering a systematic breakthrough in automated network-accelerator co-search. The educational plan is to continue and expand an existing collaboration with Technology for All, an organization that targets low-income and underserved persons, by mentoring and advising high school students from underrepresented communities. The proposed research will advance knowledge and produce scientific principles and tools for a new paradigm of designing DL accelerators with orders-of-magnitude improvement in both development speed and hardware efficiency. First, a generic design space description and a performance predictor will be developed to serve as key enablers for both (1) automated accelerator search and (2) automated network-accelerator co-search, opening up many opportunities for innovating efficient DL accelerators. Second, based on the aforementioned design space description and performance predictor, an automated and Differentiable Hardware Accelerator Search (D-HAS) engine will be designed to enable both (1) efficient navigation over the large and discrete design space of DL accelerators and (2) the significantly faster development of DL accelerators. Third, building upon the above D-HAS, an innovative network-accelerator co-search framework will be established to enable simultaneous search for optimal DL network and accelerator pairs that together will maximize the achievable hardware efficiency. Finally, a unique resource (Rice University's ASTRO platform) will be leveraged to benchmark and demonstrate the innovations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.