CAREER: Efficient and Reliable Data Transfer Services for Next Generation Research Networks

Research networks are crucial for data intensive, distributed, and collaborative science projects as they provide high speed connectivity between research and education institutions. However, users of research networks are unable to efficiently utilize available resources as existing transfer applications suffer from scalability issues at high speeds. This project designs and develops a scalable and robust data transfer framework for next-generation research networks to improve their utilization. Enhanced network performance in research networks allows seamless execution of next generation distributed science applications, thereby paving the way for breakthrough discoveries to be made swiftly. This project also promotes collaboration between scientists at geographically separated institutions by means of reducing the time it takes to share data. In addition to research contributions, this project has strong education plan tightly integrated into its research plan. The plan involves trainings for scientists to help them better utilize advanced cyberinfrastructure resources when dealing with large scale data, game development for middle and high school students to teach networking concepts, and summer schools for high school students for underrepresented groups to teach programming and networking. As trend towards data intensive distributed science continues, it is becoming increasingly important to develop data transfer services that can scale to next generation terabit per second networks and beyond. To achieve this goal, this project focuses four key research directions: First, it innovates a modular file transfer architecture to separate I/O operations from network transfers to enable dynamic and component specific tuning. Second, it implements Quality of Service support for delay sensitive distributed workflows to meet their stringent performance requirements. Third, it develops scalable, secure, and low overhead integrity verification for file transfers through in network caching/computing and probabilistic error checking mechanisms. Fourth, it integrates the developed algorithms to commonly used workflow management systems to increase its adoption by a broader science community. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.