Collaborative Research: CSR: Small: ALL-IN-ONE: Strengthening the System Aspects of Large-Scale Genomics Processing Platforms

Since the invention of DNA sequencing in 1977, genomic data has grown exponentially due to decreasing sequencing costs. Unfortunately, many bioinformatics systems lag behind in adopting state-of-the-art computing principles, resulting in wasted computing potential. Such adoption is challenging due to domain-specific expertise requirements and the limited resources available for many bioinformatics projects. Exciting research areas, including workload characterization, performance modeling, resource optimization, scheduling, and leveraging advanced hardware accelerators, remain largely unexplored in bioinformatics systems. The All-in-One (AIO) collaborative research project aims to build a next-generation genomic data processing system that incorporates state-of-the-art systems design principles. Toward this end, the project focuses on three key innovations: (1) cluster scheduling policy improvement, which uses the characterization of genomic workloads to build an execution time predictor and guide scheduling design; (2) machinery for independently-scheduled genomic tasks that support resource-aware and failure-aware directed acyclic graph-based (DAG) scheduling; and (3) a meta-compiler for a cloud-and-language agnostic processing system, which allows automated performance tuning for various domain-specific languages and cloud execution environments. The project will transfer expertise from the systems community to bioinformatics, addressing the growing computational demands for genomic data processing. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.