CAREER: Data Management for Exploratory Video Analytics

Computer vision has gone through a seismic shift over the last decade, with the availability of machine learning models capable of tackling diverse vision tasks like object detection and action localization. However, these models take a lot of time to process a video, and leveraging them for analyzing videos involves a non-trivial programming effort. There is a growing interest in the database community in designing video database management systems to tackle these efficiency and usability challenges. This project seeks to develop novel techniques for speeding up queries over large video datasets like retrieving the gameplays that lead to a touchdown in a football game. It will reduce the human labor cost of analyzing videos in a wide range of applications, ranging from urban planning to astrophysics. A close collaboration with domain scientists and industry practitioners will be pursued to help transfer the ideas to scientific and enterprise applications. The proposed work includes an integrated education plan related to video database systems in the database courses offered at Georgia Tech. This project tackles the outstanding problem of accelerating video analytics from a new standpoint. Recently proposed video database systems have two key limitations. First, their optimizers are not tailored for offline, exploratory video analytics, wasting system resources at scale and raising the cost of query processing. Second, they support a limited range of queries related to detecting objects and are unable to support richer queries like localizing actions or re-identifying objects. The goal of this proposal is to address these limitations of state-of-the-art video database systems. This work will generate knowledge of how to construct accuracy-driven optimization for complex vision pipelines. It will address the paucity of large datasets due to privacy concerns, by developing techniques for generating synthetic datasets. The innovation is not in creating new vision models, but in new database-style techniques to better optimize exploratory query workloads and to broaden the range of queries supported by video database systems. Overall, this work innovates upon the field of database systems to start a new line of research on optimizing exploratory video analytics. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.