Collaborative Research: Sensorimotor State Estimation in Human-Human and Human-Robot Interactions

Some robots that work alongside people have the special goal of reshaping how a person moves. Examples include a neurorehabilitation robot helping a patient relearn their lost movements after a stroke, or an assistive robot reducing a worker’s back strain in manufacturing. These robots must first predict how the user’s body and nervous system are engaged in the task (e.g., which muscles and parts of the nervous system are involved, and to what extent) and then optimize robotic interaction to modify those “sensorimotor states.” Current robots cannot easily do that; they either need numerous costly sensors onboard or rely on enormous datasets built for a particular task. This fundamental challenge slows the development of such robots and their widespread adoption in homes, industries, and healthcare. To overcome this challenge, this project draws on human-human interaction and studies humans’ ability to seamlessly interact and help one another with physical tasks using only the sense of touch. This fundamental knowledge gained about touch-based information exchange between humans subsequently informs the development of a new method for a robot to infer a rich set of the user's sensorimotor states solely through touch. This research’s outcomes will enable cheaper, safer, and more capable robots designed to physically assist human users and deepen scientific understanding of human movement, with far-reaching implications for neurorehabilitation and assistive robotics. The project also trains interdisciplinary researchers and openly shares all results, advancing the US robotics industry and national health and prosperity. This research develops a model-based method to estimate various sensorimotor states, from low-level muscle activity to high-level movement strategies, in real time using only haptic data (end-effector forces and motions). The computational backbone of this method is a biologically plausible yet computationally efficient sensorimotor model of the human arm that unifies neuroscientific and biomechanical theories. The benchmark for estimation is human-level performance: how well one person can infer another's sensorimotor states through haptics. Therefore, the project generates foundational knowledge of haptics-based information exchange between humans in an information-asymmetric task. The final goal of the project is to integrate the haptics-based state estimator into a model-based control framework that can deliberately manipulate a user's sensorimotor states during physical interaction, and to evaluate it against state-of-the-art methods in a user study. The resulting framework offers a generalizable, explainable, and data-efficient approach to physical human-robot interaction that communicates solely through haptics, without auxiliary sensors or user interfaces. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.