I-Corps: A Smart Context-Aware Multi-Fingered System for Dexterous Grasping

The broader impact/commercial potential of this I-Corps project is the development of a potential platform for many industries and companies, such as advanced manufacturing, e-commerce, retailer, mailing, and logistics. The platform could revolutionize and automate the process of sorting and packing irregular-shaped objects, e.g., sorting plastic mail bags with various dimensions and weights for USPS; and packing irregular-shaped goods for Walmart and Amazon. The project could also revolutionize portable automation that relies on general tools in advanced manufacturing, e.g., enabling the large footprint of aircraft assembly in confined space access for Spirit AeroSystems; and automating manufacturing processes that require various kinds of manipulation such as placing, grasping, and capping. This I-Corps project is based on the development of a platform system extending robotic grasping towards dexterous strategies for multi-fingered hands, and increasing the capability of context perception by fusing novel sensors. To accomplish dexterous grasping and manipulation, a dexterous robotic hand was designed by tightly integrating anthropomorphic gripper mechanics, context perception, and task-oriented grasp planning. The robotic hand achieves in-situ perception of object affordances by fusing multimodal sensors, including object dimensions and object characteristics such as materials, rigidity, and mass. This is the first robotic gripper that perceives object characteristics by using infrared sensors and machine-learning methods. To deploy efficient strategies for various task requirements and object affordances, a knowledge-driven model is developed to resolve the planning of dexterous manipulation motion. The model represents human knowledge as hand topology and learns task-oriented manipulation plans to adapt to different work context. Due to the integrated structure of knowledge and learning, the model produces robust and adaptive plans with a high efficiency in learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.