OAC Core: Transpass: Transpiling Parallel Task Graph Programming Models for Scientific Software

Task graph programming model (TGPM) is an essential component in many machine learning systems because it allows top-down optimization of parallelism that governs macroscale performance. Due to the distinct performance constraints exhibited by each TGPM when computing a specific machine learning workload, no single TGPM can guarantee optimal performance for all. Assisting researchers in transforming machine learning task graphs between different TGPMs is thus extremely beneficial for advancing the utilization of acceleration hardware for many of today’s machine learning-centric scientific software. Unfortunately, insufficient systems to streamline the transformation process has resulted in a significant turnaround time that prevents researchers from quickly optimizing the performance of machine learning algorithms on different computing platforms. This project creates a novel open-source system that 1) enables an automatic transformation of machine learning task graphs between different TGPMs and 2) establishes an open platform for a diverse group of researchers to contribute to program transformation research and education. This project establishes Transpass, a novel task graph-based programming environment to streamline the transformation of a program between different TGPMs with a specific focus on machine learning task graphs. Transpass introduces a new intermediate representation called control taskflow graph to represent end-to-end parallelism in a single task graph with in-graph control flow. Atop this intermediate representation, Transpass introduces a new source-to-source compiler (“transpiler”) to enable multidirectional transformations of programs between different TGPMs through transformed control taskflow graph. To support efficient execution of transformed graphs, Transpass introduces a new learning-based runtime that learns complex scheduling parameters in the given software and hardware environment. Technical contributions of this project span a multidisciplinary research community, including high-performance computing, program transformation, and machine learning. Transpass will be open-source to facilitate generalizability to many programming models and promote widespread contributions in the computing community. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.