III: Exploratory Volume Visualization via Explicit Gaussian Splatting and Natural Language Interaction

Scientific data generated by simulations, experiments, and observations are growing so rapidly that researchers often struggle to efficiently explore and understand them. Volume visualization, a key technology for examining complex three-dimensional data such as, weather systems, fluid flows, medical scans, and engineering simulations, typically requires access to large datasets and computationally-intensive processing. This project investigates a new paradigm called visualization synthesis, in which machine learning models learn how scientific visualizations appear and generate new visual representations directly, without repeatedly processing the original data. By enabling faster, more interactive, and more accessible exploration of complex datasets, the project has the potential to accelerate scientific discovery, broaden participation in data-intensive research, and improve the communication of scientific results to students, educators, and the public. Educational activities associated with the project include curriculum development, student training, outreach to K-12 educators, dissemination of open-source software, and public-facing visualization experiences that help prepare the next generation of the artificial intelligence and scientific computing workforce. This project develops a general framework for visualization synthesis based on Gaussian splatting, an emerging machine learning representation that enables efficient modeling and rendering of complex scenes. The research investigates four interconnected directions: editable Gaussian representations for inverse and interactive volume rendering; deformable Gaussian models for segmentation and feature tracking in time-varying scientific data; textured Gaussian representations for expressive visual editing and stylization of volumetric scenes; and large language model agents that support natural language interaction with visualizations. The team will design new algorithms, benchmark datasets, and evaluation methodologies to measure representation compactness, convergence speed, rendering quality, flexibility, and usability. Outcomes of the project include open-source software libraries, standardized benchmarks for language-guided visualization systems, and immersive virtual reality applications that combine visualization synthesis, scene stylization, and conversational interfaces. The resulting advances will establish foundational methods for machine learning-driven scientific visualization and enable new forms of interactive analysis, education, and public engagement. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.