CAREER: Principled System Design for Coordinated Management of Disaggregated Near-Hardware Data Processing Accelerators

Modern computing systems face increasing challenges due to the rapid growth of data-intensive applications such as machine learning, data analytics, and graph processing. Traditional approaches relying on the scaling of central processing units (CPUs) and graphics processing units (GPUs) struggle to meet high demands for memory, storage, and network bandwidth without significantly increasing data movement, energy consumption, and operational costs. The project introduces a coordinated approach that leverages low-power computational hardware accelerators, located near key resources -- such as near-memory, near-storage, and network-based solutions—to provide a scalable, high-performance, reliable, and energy-efficient framework capable of meeting future data demands across edge, datacenter, and high performance computing (HPC) systems. To achieve these goals, the project first designs an end-to-end framework to manage different accelerators by developing operating system abstractions for memory and address space management, inter-process communication, and data sharing. Second, it develops compiler and runtime support to enhance parallelism and incorporate machine learning models focused on energy efficiency, enabling effective task distribution across heterogeneous accelerators. Finally, to scale the solution across disaggregated local and remote accelerators, the project applies distributed systems principles for task scheduling and system reliability. To assess the solution's effectiveness and validate performance improvements, the project studies a wide range of applications, including graph and machine learning applications, key-value stores, data analytics frameworks, and HPC simulations such as climate modeling. By focusing on energy-efficient near-hardware accelerators and advanced software management, this project aims to accelerate datacenter, science, and healthcare applications. The project aims to leverage industrial collaborations to enhance practical impact that aligns with real-world application demands. The innovations focus on cross-layered hardware and software system design, equipping students with sought-after skills in cutting-edge technologies and a comprehensive understanding of end-to-end system design—from application development through operating system design to hardware integration. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.