CAREER: AI for the Digital Planet - Advancing Machine Learning for Satellite Remote Sensing Data for the Future

Current machine learning methods are not capable of fully analyzing and interacting with satellite remote sensing data, limiting their ability to benefit society through real-world applications. Data collected by Earth-observing satellites differ significantly from image data (e.g., photos taken with a smartphone or digital camera) or text data (e.g., online articles or social media posts). Satellite data captures the Earth’s diverse and dynamic ecosystems, environments, and human activities that are constantly changing. These patterns and changes can be subtle or obvious, large or small, difficult or easy to see with the human eye. Satellites record these patterns in many different wavelengths and sensor types, which hold much more information than the visible color wavelengths humans see. This project will advance fundamental machine learning research methods for analyzing satellite data, unlocking its untapped potential for solving societal challenges including agriculture, conservation, and natural hazards. The project will develop new technologies that improve the performance and accessibility of satellite machine learning models for different applications, thus advancing scientific progress, human and environmental sustainability, and societal welfare. This award will develop (1) a hypermodal geospatial foundation model that accommodates diverse sensor modalities and input formats, (2) a novel algorithm for zero-shot mapping using natural language prompts instead of traditional fine-tuning, (3) a testbed to evaluate model robustness under realistic distribution shifts, and (4) a zero-shot evaluation algorithm that eliminates the need for ground-truth labels. These advancements will minimize the reliance on expensive data labeling and enable flexible and efficient interaction with machine learning models. The research will be iteratively validated through real-world deployments via NASA Harvest, NASA Acres, and other user-facing organizations implementing satellite solutions for global challenges. By treating ML research and deployment as a unified approach instead of siloed steps, this project pushes model evaluation into new regimes not typically explored in machine learning research. Broader adoption of this perspective in ML research will increase end-users’ trust and adoption of machine learning research. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.