RI: Medium: Democratizing Visual AI: Enhancing Efficiency and Usability of Large Vision Models for Fostering Under-Resourced Access

This project aims to broaden access to large visual learning models for small businesses, developers and users with limited computational, data, or expert resources. Such large learned models have the potential to revolutionize multiple application areas from manufacturing to sustainability solutions to healthcare. While technology is emerging in relevance and commercial viability, it remains expensive to deploy successfully and hence its usefulness is often limited to companies and entities with maximal resources. The research from this project will focus on democratizing this technology by reducing the amount of data, computation, and human expertise needed to create and deploy application specific models. The fundamental barriers to broadened access are computationally intensive inference requirements, assumption of significant data access, and poor model interpretability. The project aims to address these challenges through an integrated plan that proposes novel learning approaches for fast model specialization with limited data, advances in inference algorithms to reduce redundancy and model reuse to increase inference and training efficiency, and corresponding advancements in visualization and deployment tools to aid usability. The project team will work together with under-resourced partners to assess the effectiveness of the new approach. While the study will focus first on computer vision models, it is expected that these advances will translate to other modalities with transformer-based models such as natural language processing and speech processing. In addition to deployment of technological advances with potential users, the findings from this project will also be integrated into undergraduate courses and research opportunities. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.