Collaborative Research: Visual Information about surface curvature from patterns of image shading and contours

Our ability to perceive the 3D shapes of objects is generally taken for granted in our day-to-day experiences, but a detailed explanation of how that is possible has eluded researchers in both human and computer vision. Whereas objects are composed of tangible substances such as wood, glass or metal, our perceptions of them are based on patterns of light. Thus, an adequate theory of perception must be able to explain how patterns of light can uniquely specify the 3D structure of the visual environment. This research project will study the visual perception of object shape, with a focus on understanding how the human visual system processes and interprets information about the three-dimensional properties of objects. A better understanding of 3D vision could provide many practical applications. For example, it could lead to dramatic improvements in the design of visually guided robots or autonomous vehicles. It could also be important for building prosthetic devices for the blind to avoid potential obstacles during locomotion or to detect the presence of slippery surfaces that create a danger of falling. The computational analysis of 3D shape from shading has been studied for over 50 years but despite the broad interdisciplinary scope it has had relatively minimal success. One possible reason is that researchers have typically simplified the problem by making unrealistic assumptions about how light behaves in the natural environment. This makes it easier to derive mathematical solutions, but severely limits the generality of the resulting analyses. In the present work, the investigators will include many of the optical effects that have typically been ignored in this domain, including cast shadows, surface inter-reflections, anisotropic reflections, light transmission, and multiple patterns of illumination. They will also perform detailed measurements of the luminance field in each condition to discover properties that are relatively invariant over changes in scene parameters. The work will focus primarily on qualitative surface features, such as bumps, dimples, ridges, valleys, and saddles, and how they correspond to similar qualitative patterns of image shading. Numerous psychophysical experiments will be conducted to document 1) how observers’ judgments of 3D surface structure are influenced by the pattern of image shading; 2) the stability of these judgments over changes in surface materials and patterns of illumination; and 3) how smooth variations of light intensity on the eye provide information about 3D shape, even when all other aspects of shading have been removed. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.