Seen and mental images in visual cortex

ABSTRACT The long term goal of our research is to understand the computational role of mental imagery in human cognition, and to operationalize this understanding to provide new tools for improving mental health. For most people, mental imagery is experienced as visual content that is independent of vision but indispensable for thought. This experience suggests that mental imagery serves an important cognitive function; however, studies of the brain systems that generate mental imagery have yet to reveal how or if mental images contribute to cognition. In this proposal, we test the hypothesis that mental imagery supports cognition by permitting the comparison of seen to unseen images. The comparison of seen and unseen images is a routine operation that occurs, for example, when one judges how a seen image differs from a remembered image, or from a target image that one wants to detect. To test this hypothesis we will measure brain activity in people as they imagine and as they complete a variety of tasks that require them to compare pictures displayed on a screen to pictures that they have been asked to remember. Using techniques borrowed from artificial intelligence (AI) we will extract from these data information about individual mental images, and then determine if this information can predict brain activity and behavior during the tasks. If successful, this proposal will establish a functional and computational role for imagery. Since very little is currently known about the function of mental imagery, our work is an essential step toward understanding how mental imagery interacts with and supports cognition, and how disregulated mental imagery can disrupt mental health.