Opening your mind’s eye: collaborating with a computer to reveal visual imagination

Minerva Schools at KGI, Stanford University
Stanford, California
Computer ScienceNeuroscience
DOI: 10.18258/7042
$2,293
Raised
327%
Funded on 6/23/16
Successfully Funded
  • $2,293
    pledged
  • 327%
    funded
  • Funded
    on 6/23/16

About This Project

Close your eyes and visualize the streets of your childhood. What do you remember of what you have seen? Now, if we both are asked to picture a street, will we imagine completely different images based in our individual experiences, or do we have a common prototype for what a street looks like? We will use an evolutionary algorithm to collaborate with human observers to visualize mental images, and analyze how idiosyncratic visual experience is reflected in the reconstructed images.

Ask the Scientists

Join The Discussion

What is the context of this research?

This study aims to create a literal picture from your visual imagination. Previous studies have successfully recovered mental representations for relatively simple categories such as letters or smiley faces. These reconstruction methods have considered images to be two-dimensional arrays of pixels. If one randomly chooses a darkness value for each pixel, then there are numLevels^nPixels possible images that can be created. For example, for even a 64x64 thumbnail-sized image with 255 possible gray levels, there are more possible images that can be created than atoms in the Universe! By contrast, we consider a scene to be represented by extended surfaces rather than pixels. This both reduces the dimensionality of of problem while creating more realistic looking images.

What is the significance of this project?

A fundamental goal for understanding human vision is to identify the mapping between image features and subsequent categorization. Our study will help us understand how personal experiences influence the mental images that we create when we think about a category such as “street”. Do different people extract the same features (common prototype), or do our visual experiences create a unique “templates”? This study will have implications for basic science, as well as potential for applications. In addition to providing critical insight into how we form complex visual categories, the technology that we are developing in this project can be used for enhancing the abilities of police sketch artists, and in architecture and design choices to make for a more legible and memorable world.

What are the goals of the project?

  • Use features of Convolutional Neural Networks to increase detail in our images.
  • Test the hypothesis that scene category representations are unique to observers by checking the extent to which reconstructions resemble scenes from three kinds of individual experience:
    • Early visual experience: the city where the individual grew up.
    • Current experience: city where the individual lives.
    • Added experience: Average of the scenes from all locations lived

Altogether, these experiments will provide critical insight into how we conceptualize complex scenes, laying the foundations for understanding how visual information is flexibly represented for recognition.

Budget

Please wait...

Our data are generated by human volunteers who are paid an honorarium of $10/hour. The experiment takes approximately 2 hours, and we anticipate recruiting 32 participants. We plan to release our published results in an open access journal so that all backers may be able to read the research. We anticipate that two papers will be generated with open access publication fees of approximately $1500 per article.

Endorsed by

I am delighted to endorse this project. The primary investigator and contributors are all extremely well qualified to address this problem, the scope of the inquiry is realistic, and the approach capitalizes on--but extends--existing scientific literature. Looking forward to the results!
This is an unusually clever approach to an old chestnut of a problem--and in my view has a high probability of producing incisive results. I recommend this project with the greatest enthusiasm!
I am really excited to endorse this project. Very interesting hypothesis and approach to tackle an important question--AND a great team!!
Smart and elegant design, with an enormous potential for better understanding how AI and human should collaborate and communicate.

Meet the Team

Michelle Greene
Michelle Greene
Associate Professor of Computational Sciences

Affiliates

Minerva Schools at KGI; Research Affiliate, Stanford University AI Lab
View Profile
Raymundo Gonzalez
Raymundo Gonzalez
Undergraduate Researcher

Affiliates

Minerva Schools at KGI
View Profile
Alex Nguyen
Alex Nguyen
Undergraduate Researcher

Affiliates

Minerva Schools at KGI
View Profile
Khang Duong
Khang Duong
Undergraduate Researcher

Affiliates

Minerva Schools at KGI
View Profile

Team Bio

Greene is a professor of Gonzalez, Duong and Nguyen. During their freshman year these students, two from Vietnam and one from Mexico, became increasingly interested in machine learning and the limitations of the ways in which humans perceive the world. Determined to use in the real world what they have learned in their classes, these students joined Michelle in an interdisciplinary project that can further understanding on human vision and satisfy the students’ desire to learn.

Michelle Greene

Michelle Greene, Ph.D. is an Associate Professor of Computational Sciences at Minerva Schools at KGI and a Research Affiliate in the Stanford University AI Lab. She earned a B.S. in Psychobiology from USC and a Ph.D in Cognitive Science from MIT, with postdoctoral training at Harvard Medical School and the Stanford University AI Lab. Her interdisciplinary research group examines questions of human visual cognition, attention, categorization, and memory.

Raymundo Gonzalez

I am an undergraduate student at the Minerva Schools at KGI. I am in my second year, double majoring in Natural Sciences and Computational Sciences. I am particularly interested in applying Physics and Mathematics to solve practical problems, but I also enjoy learning about other fields through interdisciplinary projects.

Alex Nguyen

I just finished my first year as an undergraduate at Minerva Schools at KGI, whose rigorous curriculum encourages deep engagement with the frontiers of scientific knowledge. I intend to major in Physics and Computer Science. I am particularly excited by novel computing technologies and their potential to explain fundamental questions about who we are: our ability to reason, our rich perceptual experience, and our very human emotional complexity.

Khang Duong

I am a sophomore at Minerva Schools at KGI, majoring in Artificial Intelligence. Having spent some years studying the philosophy of mind and science, I now want to use computer science to shed light on some philosophical thoughts on the subject and apply them to improve our everyday life.


Project Backers

  • 53Backers
  • 327%Funded
  • $2,293Total Donations
  • $43.26Average Donation
Please wait...