I-Corps: Translation Potential of Real-time Analytics for Petabyte-Scale Datasets and High-Velocity Data Streams

This I-Corps project focuses on the development of a digital transaction analysis system. The technology uses advanced algorithms to quickly analyze large amounts of fast-moving data. It creates real-time summaries of complex transaction data, making it possible to calculate detailed insights quickly and accurately, something that most current data analysis systems struggle to do on time. The key innovation is a set of algorithms that are proven to be accurate, fast, and efficient with memory. The algorithms are also designed to work well across multiple computers at once. This makes the approach better and faster than current systems, which often rely on basic averages or slow processing. The project provides a data analytics platform that leverages advanced algorithms to process petabyte-scale datasets and high-velocity data streams in real-time. By creating compact data summaries that retain essential information while dramatically reducing computational costs, the technology enables fraud detection systems to make accurate decisions within milliseconds while handling billions of transactions. This I-Corps project utilizes experiential learning coupled with a first-hand investigation of the industry ecosystem to assess the translation potential of the technology. This solution is based on the development of advanced algorithmic techniques that derive insights from high velocity data streams. Specifically, the technology employs mathematical constructs to create real-time summaries of massive transaction flows. This technique allows for the efficient computation of complex analytical measures. The capabilities are often infeasible for existing systems to achieve accurately and within the strict latency requirements of real-time environments. The scientific advance lies in the use of algorithms that offer provably optimal guarantees for accuracy, speed, and memory efficiency, coupled with inherent parallel computation capabilities essential for distributed processing. The approach improves on traditional methods that rely on simple statistical aggregates or slow batch processing. Users benefit from the adoption of this technology through superior automated decision-making. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.