Infrared Retinomorphic Vision

Non-technical Abstract: Bio-inspired vision represents a new paradigm in visual information processing that addresses the speed and energy efficiency shortcomings inherently present in the current state-of-the-art high-frame rate cameras. Realizing such imaging technologies using semiconductor hardware will have significant implications in various application domains, from manufacturing automation to self-driving cars. This project seeks to develop infrared vision systems that emulate the function of the biological retina by enabling the fusion of sensing and processing into one sensor element. In addition, the utilization of lead selenide semiconductors will expand the vision sensor capability toward the mid-wavelength infrared spectral region that is invisible to human eyes and mainstream semiconductor technologies. Advances from this research project will also be widely disseminated through academic courses, undergraduate research opportunities, and K–12 outreach activities, which will reach a broad student population in Newark, NJ. Technical Abstract: The goal of this research is to realize mid-infrared retinomorphic sensor devices and to demonstrate in-sensor image convolution processing via electrical programming of sensor array networks. Specifically, this research will investigate field-effect gated reconfigurable photodiodes for enabling in-sensor processing based on lead selenide, a sensor material known to be the primary choice for low-cost, uncooled mid-wavelength detector technology. Moreover, a new cross-layer-optimized retinomorphic processing engine capable of single-cycle, in-sensor processing that will enhance the image processing speed, with a low-overhead, dual-mode, reconfigurable design that will significantly reduce power consumption, will be investigated. Based on these approaches, the proposed research will generate fundamental understandings of the device physics, circuit designs, and system architecture optimizations of the mid-infrared retinomorphic vision system. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.