Collaborative Research: SHF: Small: Access Patterns in Massive Parallelism: Abstractions and Oriented Unified Memory Management for GPU Accelerated Systems

Discrete graphics processing units (GPUs) are crucial for providing the computing power of today's data centers and high performance computing systems, enabling significant advancements in many disciplines such as climate modeling, nuclear energy, drug design, social networks, deep learning, and artificial intelligence. Programming for GPUs has been laborious and error-prone due to the need to manage data migration between discrete host and GPU memories. As deep learning models and social networks become increasingly larger, and scientific workloads more data-intensive, it is imperative to relieve programmers of such tasks and make GPU programming more productive and portable. Unified Memory (UM) technologies have been developed to meet this need. Nevertheless, current UM technologies cause significant or prohibitive performance degradation. This award establishes a foundation for efficient and intelligent unified memory design, closing the prohibitive performance gap and harnessing the power of advanced GPU accelerators. It leads to reductions in time-to-production and time-to-completion of various scientific simulations and deep learning workloads, enabling them to scale up to larger problem sizes with ease. The award includes rich education, outreach, and broadening participation activities, offering in-class and out-of-class experiences, as well as team-based undergraduate research and engaged learning spanning multiple semesters. It recruits and supports students from underrepresented groups, including people with disabilities, fostering their inclusion and belonging. The comprehensive framework, ACCess Pattern ORienteD (ACCORD), includes abstraction methodologies, cost models, and techniques to enable efficient UM algorithms and systems for various workloads and problem sizes. Its key innovation lies in the abstraction of access patterns, which uses metrics obtainable at the system level to capture the spatial distribution and temporal repetition patterns of massively parallel memory accesses. This abstraction empowers the quantitative assessment of their interaction with UM designs and guides the optimization of algorithms and UM techniques to eliminate performance bottlenecks and optimize data movement effectively. The research objectives include: (1) devising the abstraction of access patterns for UM-based GPU-accelerated systems, (2) developing quantitative methods to analyze the cost of various access patterns and their interaction with UM techniques, (3) designing access pattern-oriented UM techniques for online deployment, and (4) integrating ACCORD into real-world UM systems to support various applications. This project is jointly funded by the Software and Hardware Foundations (SHF) core program at the Division of Computing and Communication Foundations (CCF) and the Established Program to Stimulate Competitive Research (EPSCoR). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.