CAREER: Rethinking PIM-Assisted GPU Computing for Multi-Tenant Artificial Intelligence

Artificial intelligence (AI) systems have entered the “multi-tenant” era, where multiple deep neural network (DNN) models are executed simultaneously. This involves concurrent deployment, computation, and interaction of multiple DNN models, increasing computational complexity and triggering new challenges: (1) How can a scalable and flexible computing architecture be realized that can adaptively host heterogeneous and concurrent DNN models? (2) How can computing flexibility requirements in multi-tenant DNN scenarios be met? (3) How can an efficient, end-to-end toolchain for building next-generation AI applications be realized in this context? This project addresses these challenges through three research thrusts: Thrust 1 investigates a novel processing-in-memory (PIM)-assisted graphics processing unit (GPU) architecture with innovative multi-tenant support, addressing important resource contention and model interaction issues. Thrust 2 explores dedicated GPU- and PIM-oriented scheduling techniques to enhance the platform’s performance. Finally, thrust 3 further enhances the multi-tenant AI application development cycle with algorithm optimization and code deployment support. With the successful completion of these thrusts, this project can achieve breakthroughs in modern AI computing and support the next generation of AI applications. The proposed techniques have the potential to accelerate AI design and deployment, spurring even wider AI utilization. This can contribute to important application areas with societal importance, including autonomous driving, metaverse immersion, smart agriculture, and industrial infrastructure. This project will also benefit students --and by consequence, society-- by incorporating research results within relevant courses, increasing the participation of women and other underrepresented groups in computing, and sharing research results with researchers, companies, and government agencies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.