CAREER: Context-Aware Resource Efficient Natural-Language Queries over Longitudinal Wearable Sensor Data

Wearable devices such as fitness trackers and smartwatches collect large amounts of information about movement, sound, and the body, but the tools that come with them usually show only simple charts that do not answer the questions people actually have. This project builds a system that lets anyone ask everyday questions in plain language, such as how well a baby slept last week or whether a night-shift worker became too warm, and receive clear answers drawn from the sensor information. The system is designed to perform its processing primarily on smartphones, wearables, or other everyday devices, so it responds quickly and keeps personal information on the device rather than always uploading it to the cloud. The research is organized into three connected thrusts. The first thrust develops a way to store long histories of sensor data on a single device by saving information in compact form and at several levels of detail, so that both broad trends and brief moments can be recovered when a question calls for them. The second thrust builds a reasoning layer that weighs each sensor stream according to the question being asked and settles cases where two signals point to different conclusions. The third thrust designs a runtime that adjusts how much computation each question requires, using early stopping, trimming of unneeded work, and selective handoff of difficult cases to the cloud when local limits are reached. The work makes information from sensors useful to people who are not technical experts, including parents, patients, caregivers, and frontline workers, and it supports settings such as home care, rural clinics, and school programs where computing resources are limited. The real-world usability of the system will be demonstrated with data from an infant behavior study, showing how it helps families understand daily routines. To support reuse and community progress, open datasets, models, evaluation tools, and course materials will be released for public use, and an annual open data challenge will invite undergraduates, high school students, and researchers at under-resourced institutions to take part. Summer workshops for high school students and participation in national research programs for undergraduates and teachers will help strengthen the science and engineering workforce. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.