EAGER: Exploring Robot-K9 Teaming through Canine Cognitive Modeling and Multimodal Communication

Canines and humans team together in a variety of situations, with dogs providing assistance and companionship to the elderly and disabled, detecting explosives and illicit drugs for law enforcement, and herding and protecting livestock in agriculture, among many other roles. Self-sufficient canine-robot teams offer unique advantages over canine-human teams, for example, by allowing human operators to remotely monitor multiple agents from a distance, thus cutting costs and decreasing human risk. This EArly-concept Grant for Exploratory Research (EAGER) project will build a foundation for ongoing canine-robot research by establishing bidirectional communication between robots and dogs. First, the project will establish practical procedures for training dogs to reliably respond to robot teammates. This thrust will explore a variety of potential information channels, including synthesized human speech, abstract audio codes such as whistles or ultrasound patterns, tactile cues delivered through buzzers mounted on a wearable harness, and gestures and expressions using humanoid robots and human-like robot faces. Second, the project will use artificial intelligence (AI) and machine learning (ML) to create foundation models for canine cognitive state, allowing robot team members to autonomously assess canine readiness, stress, and task engagement in real time, and to understand and interpret postural, behavioral, and audible cues. Effective autonomous canine-robot teams will mitigate risks for humans in dangerous disaster areas, such as those involving radiation or toxic chemicals. The project also includes specific activities for stakeholder engagement and workforce development. The specific technical goals of the project are (1) creating machine learning foundation models for robot interpretation of canine cognitive state, including algorithms for canine behavior recognition, safety/risk prediction, and behavior classification; and (2) extending methods in probabilistic sensor fusion and stochastic optimization to heterogeneous uncrewed robot-dog teams, a unique biological-technological system for dynamic sensing in uncertain environments. The project will include experimental comparisons of the response of canine brains to commands from human handlers versus commands delivered by robots. The characterizations will include multimodal interfaces such as electroencephalogram (EEG) and video. Commands will be presented using a human voice (e.g., “fetch”), as well as ultrasound and vibrotactile cues (e.g., “buzzing” the left portion of the vibrotactile sensors to command the dog to go left). Through these experiments, the project will generate a new multimodal dataset of canine behavior, integrating audio cues (e.g., barking signatures), kinematic patterns (sitting, retreating, approach), and physiological and emotional signals (e.g., EEG-based stress/engagement indicators) that will be useful not only for canine cognitive foundation models but also more generally for cognitive and behavioral scientists and organismal biologists. This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.