CAREER: Towards Real-Time Polarimetric Synthesis from Probabilistic Representations

A way to extend the human eye’s capability is to create a fuller view of an object than is available in a traditional camera. This can be achieved via polarimetric imaging, which considers polarization of light that corresponds to every pixel in an image to create a better image of the object. Polarization of light can be thought of as compartmentalizing the electromagnetic light wave into subcomponents. These subcomponents are used by polarimetric cameras to reveal information that is normally not available in regular cameras. With the introduction of commercial polarimetric cameras in 2018, there has been a growing interest in polarization for computer graphics, physics-based modeling, and vision algorithms. Common indoor environments exhibit relatively small polarization attributes that can still provide valuable information. These can be modeled computationally and can be used to augment the capabilities provided by traditional cameras. This project envisions simulating the polarization attributes of indoor environments to answer the question: “If polarization sensitivity is added to webcams, conferencing capture systems, and cellphone cameras, what new capabilities will be possible?” This project will provide the research community with an open-source polarization image library and data synthesis pipeline to demonstrate our novel computational approaches, which can be adapted for vision, graphics, and the design of capture systems in the pursuit of informatics from commercial polarimetry. Training researchers, including those from Native American communities, for entrepreneurial pursuits at the intersection of computational and optical sciences is a legacy goal of this research. This education plan will leverage the University of Arizona’s exemplary student resources. To increase students’ awareness of the economic opportunities made possible by their education, entrepreneurship will be valued and practiced in research activities. This research will investigate from three perspectives: mathematical (e.g., inverse problems), computational (e.g., Monte Carlo ray tracing), and optical physics (e.g., appropriate physical assumptions for commonplace indoor materials). Current simulations rely on generic representations of polarimetric light-matter interactions that are overly complex and lack insight from empirical observations. This research will investigate fundamental changes to computing polarized light-matter interactions that are: (i) compressed by a factor of two given simplifications appropriate for many indoor materials; (ii) physically constrained when interpolated or averaged; (iii) compatible with quaternion rotation; and (iv) statistically evaluated using a representation that facilitates importance sampling in the polarimetric domain. The improved accuracy and computational efficiency of our proposed approach will enable unprecedented exploration of plausible inverse solutions. Our library of indoor polarimetry will be a new resource for data-driven learning methods and a long-term impact of this research. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.