About This Project

Close your eyes and visualize the streets of your childhood. What do you remember of what you have seen? Now, if we both are asked to picture a street, will we imagine completely different images based in our individual experiences, or do we have a common prototype for what a street looks like? We will use an evolutionary algorithm to collaborate with human observers to visualize mental images, and analyze how idiosyncratic visual experience is reflected in the reconstructed images.

Ask the Scientists

What is the context of this research?

This study aims to create a literal picture from your visual imagination. Previous studies have successfully recovered mental representations for relatively simple categories such as letters or smiley faces. These reconstruction methods have considered images to be two-dimensional arrays of pixels. If one randomly chooses a darkness value for each pixel, then there are numLevels^nPixels possible images that can be created. For example, for even a 64x64 thumbnail-sized image with 255 possible gray levels, there are more possible images that can be created than atoms in the Universe! By contrast, we consider a scene to be represented by extended surfaces rather than pixels. This both reduces the dimensionality of of problem while creating more realistic looking images.

What is the significance of this project?

A fundamental goal for understanding human vision is to identify the mapping between image features and subsequent categorization. Our study will help us understand how personal experiences influence the mental images that we create when we think about a category such as “street”. Do different people extract the same features (common prototype), or do our visual experiences create a unique “templates”? This study will have implications for basic science, as well as potential for applications. In addition to providing critical insight into how we form complex visual categories, the technology that we are developing in this project can be used for enhancing the abilities of police sketch artists, and in architecture and design choices to make for a more legible and memorable world.

What are the goals of the project?

• Use features of Convolutional Neural Networks to increase detail in our images.
• Test the hypothesis that scene category representations are unique to observers by checking the extent to which reconstructions resemble scenes from three kinds of individual experience:
  • Early visual experience: the city where the individual grew up.
  • Current experience: city where the individual lives.
  • Added experience: Average of the scenes from all locations lived

Altogether, these experiments will provide critical insight into how we conceptualize complex scenes, laying the foundations for understanding how visual information is flexibly represented for recognition.