Collaborative Research: Frameworks: Diamond: Democratizing Large Neural Network Model Training for Science

Diamond is a service designed to democratize access to cutting-edge DL methods by abstracting the use of HPC resources. Diamond combines novel computer science research with translational computer science to reduce the significant barriers that impede adoption of DL methods in science. With Diamond, domain scientists can focus on the neural network architecture design to solve their domain-specific challenges without worrying about Cyberinfrastructure management. Diamond also contributes to key educational outcomes. PhD students work directly on project goals, and tools developed in the project will be used in undergraduate and graduate-level courses. The tools will also be used in summer schools and programs at TACC, UChicago, and NCSA. Targeted recruitment of students from underserved communities at the graduate, undergraduate, and high-school levels will address diversity and outreach goals. Diamond builds upon prior work in software ecosystem management, parallel computing, deep learning, and data management, combining disparate capabilities into a cohesive and user-friendly framework. It provides a web service-enabled programming interface supporting the DL lifecycle from development to deployment and dissemination. It offers container configuration, automatic scaling for distributed training, hyper-parameter tuning, and model sharing. It also applies crucial performance optimizations, including planning for long training jobs, performance-aware model placement, cross-cluster training, and data management. Diamond results are made available to domain scientists, computer scientists, and engineers supporting DL applications in HPC centers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.