Collaborative Research: Physics-Empowered Vision-Based Tactile Gel-Robots for Multi-Physical Perception and Ultra-Gentle Manipulation

This collaborative research project will enable new capabilities for robots to perceive and manipulate soft and fragile objects in applications such as surgery. A central innovation is a thin gel coating that changes its optical properties directly in response to external forces. These optical changes reveal the distribution of forces across the coating in colorful fringe patterns that appear on the surface, even when the applied forces are too small to significantly deform the coating shape. Specifically, this project will create tactile-based robots that integrate the color-changing gel into a force-interpreting optical system, giving the robot the capability to perceive mechanical and physical properties of soft and fragile objects and manipulate these objects without damage. This advancement will surpass existing tactile robots in areas such as medical robotics, assistive technologies, and mixed and virtual reality. In addition, this project will establish a unique platform for workforce development through educational and training activities in robotics and provide an inclusive avenue for engaging underrepresented groups in STEM disciplines. This project will integrate fatigue-resistant photoelastic gel into a stress-interpreting optical system for high-performance vision-based tactile gel-robots that can obtain multi-physical perception and execute ultra-gentle manipulation of soft and fragile objects. Specifically, this project will leverage the molecular design of fatigue-resistant photoelastic gels, the mechanical design of a stress-interpreting photometry system, and the algorithm design of physics-informed machine learning to perceive, visualize, and interpret robot-object interactions. Finally, this project will integrate material design, mechanical design, and algorithm design to build a physics-empowered, vision-based tactile gel-robot, and demonstrate robotic multi-physical perception and ultra-gentle handling capabilities previously unattainable (for example collecting fragile jellies for study in marine biology or cutting and manipulating foods like custards in assistive robotics). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.