Collaborative Research: CSR: Co-Designing Integrated AI-Enabled Compiler-Runtime Systems for Future HPC Architectures

Modern supercomputers combine diverse processors and accelerators but their software relies on incompatible layers (e.g., MPI, OpenMP, CUDA, HIP). This fragmentation creates problems: optimizations in one component break others, and developers must manually coordinate work distribution, memory allocation, and data movement across systems. This project aims at creating a unified control system that coordinates scheduling, memory placement, and communication across the entire hardware-software stack. Rather than requiring complete redesign, existing program components connect to a single coordinating substrate. New accelerators can be integrated without rewriting applications. Built on HPX, an established supercomputing platform, this approach promises more manageable, efficient supercomputer software. The HeliX project seeks to provide fundamental new understanding about how to co-design the interfaces and feedback loops between programming model, compiler, runtime, and heterogeneous backends so that scheduling, placement, and communication decisions are coordinated end-to-end. The central goal is to identify what information must cross each boundary, which knobs matter at which decision points, and what control strategies (rules, variants, just-in-time compilation, and learning) deliver stable, low-overhead performance portability. The project pursues this goal by (i) co-designing a compiler-visible library and embedded domain-specific language programming surface for expressing asynchrony and intent (constructs and metadata annotations); (ii) extending compilers so these constructs and annotations are recognized and used to emit persistent intent metadata and variant families; (iii) establishing metadata and telemetry as first-class, bidirectional coordination channels between compiler, runtime, and specialization; (iv) using the integrated runtime as a control plane that consumes intent and telemetry to coordinate scheduling, placement, and memory/communication decisions across CPU/GPU/network backends; and (v) closing the loop with explicit machine learning coordination that steers safe tuning, code variant selection, and on-demand code specialization when fixed heuristics break down. All artifacts will be released under a permissive license, with upstream contributions to HPX and the LLVM compiler framework where appropriate, enabling immediate reuse in research and production settings. HeliX will provide migration guides, reference examples, and performance diagnostics that lower the barrier for scientists and engineers transitioning from existing fragmented codebases to integrated, adaptive execution. The project will develop tutorials and reusable classroom modules on compiler-runtime co-design and data-movement-aware execution, emphasize inclusive mentoring and outreach to broaden participation in high-performance computing systems research, and publish evaluation scripts to support reproducible comparisons. Where appropriate, HeliX will engage community processes (e.g., via the High-Performance Software Foundation) to maximize sustainability and interoperability of the project’s interfaces and metadata conventions. The lead institution is in an EPSCoR state, and the project will create training opportunities for students and postdoctoral researchers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.