Collaborative Research: SHF: FlexScale: Exploiting Workload Elasticity to Increase HPC Throughput while Lowering Cost via Dynamic Power Steering

High-performance computing (HPC) facilities are critical infrastructure for scientific discovery, engineering design, and large-scale artificial intelligence (AI). However, these facilities require power budgets in the tens of megawatts and face growing pressure from limited grid capacity, rising electricity costs, and variable renewable energy supply. Today’s HPC systems are largely managed using fixed power budgets and rigid job allocations, leaving substantial opportunities for improved performance, energy efficiency, and cost savings untapped. This project develops FlexScale, a new approach for operating HPC systems under changing power availability and electricity prices. FlexScale helps computing centers make better use of available power by directing it where it is most needed and adjusting less urgent or more flexible work when power is limited or expensive. By helping HPC centers do more scientific and AI computation with existing power infrastructure, FlexScale supports more affordable and scalable national cyberinfrastructure. FlexScale establishes new abstractions, models, and algorithms for coordinating job elasticity, scheduling, resource allocation, and power control under time-varying power availability and electricity cost. FlexScale introduces innovations in: (1) workload classification by priority class, execution elasticity, and performance requirements; (2) fine-grained management of malleable jobs through resource expansion, shrinkage, and power control under varying power constraints; (3) application-agnostic power shifting to reduce critical-path execution time while respecting hierarchical power limits; and (4) safe hardware overprovisioning to increase job capacity under existing power budgets. The project evaluates these methods using representative HPC and AI workloads and integrates the resulting approaches with open-source resource-management software. The expected contribution is a principled and practical foundation for power-adaptive HPC scheduling that improves throughput, utilization, and cost effectiveness under real-world power constraints. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.