SHF: Small: Taming Huge Page Problems for Memory Bulk Operations Using a Hardware/Software Co-Design Approach

Recent advances in non-volatile memory (NVM) technologies promise significantly higher capacities compared to traditional dynamic random-access memory (DRAM). Today’s computers often combine NVM with DRAM to construct a unified main memory architecture that meets the extremely high demand of many emerging big data and artificial intelligence applications. However, managing ultra-high capacity memories introduces daunting systems challenges. In particular, the massive parallelism of NVM drives adoption of huge two megabytes or even larger memory pages. The successful completion of this project's activity will be a significant step towards accelerating the secure non-volatile memory architectures by orders of magnitude performance improvement. This project will also contribute to society through engaging under-represented groups from a Hispanic Serving Institution and research dissemination for computer science and engineering education and training. The research project tackles these issues using a hardware and software co-design methodology. In this project, a general-purpose, fine­-grained memory management system will be designed, modeled, and implemented to significantly improve both the speed and bandwidth of single-level, DRAM combined with ultra-low latency, non-volatile memory architecture for both central processing unit (CPU) and graphic processing unit (GPU) machines, in particular for huge page configurations. The unused and overprovisioned parts of existing hardware architecture units will be exploited via both non-intrusive and intrusive ways to enable fine-granularity memory management, and achieve high efficiency and effectiveness of the Copy-on-Write mechanism. Several new Copy-on-Write use cases, including the checkpointing and snapshot, and large language model serving will be further studied. Furthermore, the new memory architecture will be integrated with the high-speed processing workflow of big data management, artificial intelligence programs, and hardware security machines. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.