IMR: MM-1C: An Extensible Platform for Asking Research Questions of High-Speed Network Links

Over the past decade, network speeds have increased dramatically, outpacing the tools that both researchers and operators used for understanding and securing networks. Beyond volume, Internet traffic itself has also fundamentally changed — most traffic today is encrypted and relies on cloud infrastructure or content distribution networks. Visibility into Internet traffic is critical for understanding performance characteristics, troubleshooting operational issues, performing Internet Measurement Research, and answering fundamental security questions. Many tools have been proposed for regaining network visibility, but few have seen widespread adoptions due to fundamental limitations in the types of questions they were capable of asking. Rephrased, proposed tools were fast, but could not accommodate the complex questions that researchers and operators need to ask in practice. This project builds an open source network measurement platform that is specifically designed to allow asking complex, real-world questions of networks, as defined by a broad set of researchers and ISP operators. The platform runs on general purpose (x86) servers and allows users to subscribe to subsets of parsed network traffic needed for their analysis. In contrast to prior work, the platform is not constrained to a custom query language, but instead builds on the flexibility, performance, and security provided by the Rust programming language. The project’s success will enable researchers to more easily pursue Internet Measurement and Security research as enable operators to better understand and secure their own networks. To ensure the platform’s long term usability, the research team will additionally partner with external research groups to understand the diverse set of operational constraints present in production networks and to deploy the measurement platform in those environments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.