Variable Stiffness Robotic Hands with Intrinsic Force and Slip Sensing based on Event Vision and Machine Learning for Dexterous Grasping and Manipulation

This project advances the design and functionality of robotic hands by developing an integrated framework that enables more dexterous and intelligent grasping. Robotic hands are essential for enabling robots to interact with the world across industries such as manufacturing, healthcare, and service robotics. However, most current designs are limited in functionality, often focusing on a single capability such as flexibility or sensing. This award supports research activity that introduces a new approach that combines mechanically intelligent finger structures with embedded force, slip, and vision sensing based on event-driven cameras. The resulting technology will enable robotic hands to perform both delicate and power grasps while adapting to changing environments. The research will have broad impacts on industry through the development of smart robotic grippers and will promote STEM education through hands-on outreach activities with K–12 students, as well as integration into university-level robotics design curricula. These efforts aim to inspire the next generation of engineers and support the US workforce in high-tech sectors. Technically, the project introduces a novel design methodology for variable-stiffness robotic hands using a double-layered compliant mechanism structure that supports dynamic stiffness control and integrated sensing. The research team will use event-based vision and machine learning to extract force and slip information from finger deformations without the need for traditional tactile sensors. Key outcomes include (1) analytical and learning-based models for mechanism design, (2) vision-based force and slip detection algorithms, and (3) closed-loop control methods for stable, adaptive grasping. The system will be validated through hardware implementation and simulation. The integrated platform—including hardware prototypes, control software, and experimental data—will be shared with the broader research community. This work contributes to the field of compliant robotics and intelligent manipulation and lays the foundation for future developments in human-level robotic hand systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.