Restoring complex movement and locomotion after paralysis through collaborative copilots

PROJECT SUMMARY There are no widespread devices available that significantly improve the quality of life for those with paralysis. A widespread and effective device for paralysis should be low risk, yet high performance, enabling complex movements needed to perform everyday tasks. We propose to achieve a low risk, high performance device through three thrusts. Our first thrust is to capitalize on advances in artificial intelligence (AI) that enable us to train robots that perform the same types of tasks that humans perform, including folding laundry, putting food in a pan, opening door knobs, or picking up a bag of chips. Because robots can perform these tasks, we reason that as long as an AI copilot can infer a human's goal to perform a certain movement, the copilot can then help perform the complex movements associated with the task. We achieve this by training new copilot architectures that use computer vision, as well as non-invasive signals reflecting the user's intent, to perform complex actions. Our second thrust is to take advantage of and fuse many non-invasive signal sources, which together provide enough information for the copilot to infer the user's goal and help the user complete an action. Our third thrust is to develop methods to adapt the user and copilot as they share control. We hypothesize this will have a key impact on increasing system performance and robustness. We believe this proposed work, if successful, will have a transformative impact on the development of clinically viable devices that help people with paralysis move autonomously.