RI: Small: Weak 3D Cameras

This project proposes a new class of vision systems called weak 3D cameras, which are designed to recover simplified geometric representations of the environment that are sufficient for fast decision-making but far more efficient than conventional dense 3D image reconstructions. While traditional 3D cameras aim to build detailed models of a scene, many real-world applications—such as drone navigation, robotic manipulation, and augmented reality—often require only partial or approximate geometry to function effectively. These applications also operate under tight constraints on power, compute, and latency, making dense 3D sensing impractical. This project explores the design of vision systems that can perceive their surroundings through simpler, faster, and more robust representations. The research builds on two new kinds of geometric representations—Inertial Safety Maps (ISM) and Blocks World Sets (BWS)—which can be captured directly using low-cost hardware and lightweight algorithms. The project is organized into three thrusts: building compact cameras that work reliably in challenging real-world conditions; enhancing them with additional low-cost sensors and modern geometric priors and world models to improve performance; and extending their use beyond static geometry to understand motion and semantics. The goal of the project is to create fast, affordable 3D vision for the next generation of intelligent machines, with potential uses in robotics, wearable tech, disaster response, and more. The outcome of this work will enable fast, low-power 3D perception on compact platforms, with potential impact across embedded vision, augmented reality, and assistive technologies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.