NeTS: Medium: SLATE: Service Layer Traffic Engineering

Most people use online applications -- web sites, maps, videoconferencing, shopping, messaging, and more -- on their phones and laptops every day. These applications are hosted in clouds, meaning that important parts of their application logic, databases, and storage run on servers in large data centers, typically in multiple sites across the country or world. Such applications usually involve many distinct components of software performing dozens or hundreds of small tasks behind the scenes to collectively produce the result the user wants, like a web page or a high-quality video stream. Because of the complexity of these cloud-hosted applications and the unpredictable workload they receive from users, it is difficult to optimize their performance and resource usage. The typical result today is that applications' resources are overprovisioned -- meaning, they are allocated much more computing resources than necessary; 70% waste, or even more, is common. Current methods to deal with this problem try to plan better long-term resource allocations, but do not help with short-term workload changes and do not help with network-related costs. This project will design and develop a system called Service Layer Traffic Engineering (SLATE). SLATE will help cloud-hosted applications to optimize the performance of tasks, deal gracefully with short-term workload fluctuations with less resource waste, and optimize cloud bandwidth and computing costs. At a technical level, SLATE will provide easily usable optimization for modern cloud-hosted applications with a new optimization layer beneath the application. SLATE focuses on applications which are microservice-based: such applications split each task into many small components which run on different servers. SLATE will extend existing open-source service meshes, which today provide networking functionality for microservice-based applications. However, this involves significant new challenges. First, the project will develop techniques to prioritize requests automatically across bottlenecks spanning multiple resource types (computing and network capacity). Second, the project will design methods to route requests in real time among possible servers, intelligently trading off latency, cost, bandwidth, and outlier considerations in multi-cluster environments. Third, the project will develop a theoretically grounded approach to decompose the service layer traffic engineering problem into a decentralized design, with local decisions enabling fast reaction, and just enough global coordination to achieve optimality. The project will seek to achieve substantial broader impact by benefiting cloud-hosted applications with significant performance, resource utilization, and cost improvements. The project will also support activities to broaden participation in computing, including mentoring Research Experiences for Undergraduates (REU) students, attracting applicants from underrepresented groups, and participating in the EECS Rising Stars workshops. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.