Collaborative Research: CPS: Medium: Efficient Human-in-the-loop Learning for Human-centric Cyber-Physical Systems

Human-centric cyber-physical systems (CPS) such as assistive driving and exoskeletons aim to augment human capabilities instead of replacing humans. Humans can collaborate with these machines to overcome corner cases and demonstrate the correct action under safety-critical situations. Such human collaboration enables the human-centric CPS to achieve a better outcome than either could achieve alone. In this project, investigators will develop an efficient human-in-the-loop learning framework for human-centric CPS. During training, the machine will learn to make decisions in an uncertain environment, while the human will oversee the machine and actively intervene when anomalous or unsafe behavior occurs. The human will then demonstrate the correct action to the machine. The project’s novelties are incorporating a human subject to guard the learning agent, where the human can actively intervene in unsafe situations and demonstrate the correct actions to the agent during training, and developing a reward-free learning approach that substantially encourages learning efficiency, safety, and AI alignment. The proposed human-in-the-loop learning framework is being instantiated in two human-centric CPS including assistive driving and exoskeleton. The project's impacts are facilitating harmonious human-machine collaborations and enabling CPS's efficient and safe autonomy. This research contributes to establishing best practices and standards applicable to various industries where it is essential to integrate humans in the operation of CPS, including automotive, package delivery, and rehabilitation. The team is creating research and training opportunities for high school, undergraduate, and graduate students in machine learning, robotics, control, and biomechanics. The project breaks away from the prevailing paradigms of model-based control and safe reinforcement learning through three research thrusts. 1) Development of a human-in-the-loop learning framework that incorporates a human subject to guard the learning agent, where the human can actively intervene in unsafe situations and demonstrate the correct actions to the agent during training. This approach is reward-free and encourages learning efficiency, safety, and AI alignment. 2) Creation of digital twins of task-specific human behaviors for evaluating the proposed learning method in each targeted CPS, with focus on developing a simulated environment of human behaviors in driving and exoskeleton. 3) Empirical evaluation and real-world experimentation of each targeted CPS to train and evaluate the proposed learning methods against various scenarios in simulation and real-world settings to validate their performance. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.