CSR:Medium: A Cross-stack Approach to Reduce Memory Carbon in Cloud Data Centers

The scaling of memory capacity (i.e., bits per memory chip) increasingly lags behind the density scaling of other system components (e.g., cores per area in a CPU chip). As a result, a typical cloud server today contains 100s of memory chips, which may increase to 1000s under the current scaling trend; manufacturing and powering so many memory chips per server will be environmentally unsustainable. This project will explore how to co-design hardware and software to store values more densely in memory and reduce how much memory to manufacture and power to reduce the carbon footprint of future cloud data centers. While prior art has explored memory compression in hardware, they have only explored how to do so in the most rudimentary software scenarios - natively running a single program that accesses little to nothing beyond memory. The software stack in cloud is much more complex; user applications run in virtual machines, concurrently with collocated workloads, and often heavily exercise the operating system (OS) file cache and other in-memory caches. This project - CloudComp- will co-design hardware memory compression with different layers of the cloud system software (e.g., hypervisor, storage stack, in-memory databases, job scheduler) to enable practical deployment that can satisfy the diverse requirements and application scenarios in cloud. CloudComp will bring together researchers in computer architecture, cloud computing, OS, storage systems, and databases. To facilitate real world impact, this project will build and release real-system prototypes of hardware memory compression and partner closely with industry. By enabling an alternative path to scale up the effective size of the memory size for future cloud data centers, CloudComp will help reduce the impact of cloud computing on climate change over the brute-force approach of making more memory chips. Densely storing more data into the available amount of memory can also benefit climate in other ways such as enabling bigger-scale and/or finer-resolution climate modeling and simulation. CloudComp includes educational and engagement activities to broaden participation in research and attract new students, especially seeking out students from underrepresented groups. CloudComp will actively involve undergraduate students in building real-system prototypes to cultivate their curiosity for research. Lastly, the cross-layer insights gained through CloudComp will help guide the research of other complementary memory system techniques to combat the slowing physical scaling of memory. CloudComp is funded in part by the National Discovery Cloud for Climate (NDC-C) program as a core purpose of the project is to reduce the carbon emissions of cloud systems through this research. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.