STTR Phase I: Semantically-Enabled Augmented Reality for Manufacturing

This Small Business Technology Transfer (STTR) Phase I project facilitates safer and more efficient human-centered manufacturing tasks. The introduction of context-sensitive work guidance through immersive technologies will expedite workforce training, enhance users' spatial awareness, and outperform existing manufacturing work instruction systems, leading to heightened productivity across industries. This development embodies the emergence of cyber-human relationships and Digital Twin and Smart Factory applications, reinforcing U.S. manufacturing leadership, bolstering economic competitiveness, and fortifying national security. The anticipated commercial Platform-as-a-Service (PaaS) solution is poised to benefit approximately 10,000 U.S. manufacturing firms. Beyond its economic implications, the first-generation, open-specification Reality Modeling Language (RML) developed in this project is expected to gain widespread acceptance in the international standards community, improving spatial system automation across diverse industry verticals. Ultimately, this system will render the physical world more accessible, searchable, and comprehensively annotated with data, unlocking new frontiers in user support, safety, and efficiency. This Small Business Technology Transfer (STTR) Phase I project addresses mission-critical challenges for fully leveraging Augmented Reality (AR) tools in manufacturing environments. It draws upon ontologically structured data and a proprietary Artificial Intelligence (AI)-driven knowledge system for automating the generation and display of context-specific AR content in 3D space, eliminating the need for individually designed AR interactions. The solution enables training and work instruction systems to become spatially- and contextually aware, in order to adapt to dynamic conditions impacting worker safety and efficiency. The objective of this project is to demonstrate and quantify how automatically generated, spatially- and semantically aware AR can provide work guidance, machine status data, and hazard warnings to increase worker capabilities versus conventional guidance tools. The RML will be derived and logically describe and computationally code the 3D spatial scene of a simulated factory floor, and later, RML will be released as an open code library to the developer community. The system will sense the real world and objects in real-time, learn as input is received, and prioritize and render AR content communicating context-specific suggestions and warnings. This project will demonstrate integration between workers, their environment, and the tools engaged to complete their tasks so production personnel can act confidently, safely and effectively. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.