CPS: Medium: Aerial Co-Workers: Augmenting Physical and Cognitive Human Capabilities

This project studies the algorithmic foundations and methodological frameworks to augment human capabilities via a novel form of physical and cognitive collaboration between human and multi-agent robotic systems, creating Aerial Co-Workers. These machines will actively collaborate with each other and with humans and tackle the fundamental gaps related to human-MAV collaboration at both physical and cognitive levels. The project is organized along two main thrust areas: Physical Collaboration and Cognitive Collaboration. The first thrust aims to significantly augment the physical ability of human workers by taking advantage of physical collaboration between the operator and a network of interconnected quadrotors, equipped with a set of flying hands", transporting objects. This will produce novel scientific solutions for human-robot collaborations to account for the complex legibility of the motions, and the variability of the relative positions of the agents. The second thrust aims to address two perception consensus problems to enable MAV-assisted augmented reality (AR) to augment the cognitive ability of operator(s). The key is to consistently collect, analyze, and display contextual information via multiple MAVs for effective and natural human-robot visual interactions. Aerial Co-Workers will get vantage viewpoints of the environment occluded from the humans which can be customized and augmented directly in the workspace to facilitate human actions via novel metric-semantic collaborative space mapping. This project will have a strong societal impact as a disruptive technology for industry as well as the construction market, which is in urgent need of innovative solutions for enhancing the efficacy while maximizing safety. The outcome will enable safer, faster, and simpler task execution in scenarios including maintenance, inspection, transportation, and search and rescue. The project will contribute to lowering the barriers for new researchers in robotics, computer vision, and machine learning by making hardware designs, algorithms, datasets, and code available on open-source forums. The playful nature of AR tools and quadrotors employed in this project will contribute to engaging K-12 and undergraduate audiences. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.