Simple line drawings suffice for functional MRI decoding of natural scene categories

Dirk B. Walther, Barry Chai, Eamon Caddigan, Diane M. Beck, Li Fei-Fei

Research output: Contribution to journalArticle

Abstract

Humans are remarkably efficient at categorizing natural scenes. In fact, scene categories can be decoded from functional MRI (fMRI) data throughout the ventral visual cortex, including the primary visual cortex, the parahippocampal place area (PPA), and the retrosplenial cortex (RSC). Here we ask whether, and where, we can still decode scene category if we reduce the scenes to mere lines. We collected fMRI data while participants viewed photographs and line drawings of beaches, city streets, forests, highways, mountains, and offices. Despite the marked difference in scene statistics, we were able to decode scene category from fMRI data for line drawings just as well as from activity for color photographs, in primary visual cortex through PPA and RSC. Even more remarkably, in PPA and RSC, error patterns for decoding from line drawings were very similar to those from color photographs. These data suggest that, in these regions, the information used to distinguish scene category is similar for line drawings and photographs. To determine the relative contributions of local and global structure to the human ability to categorize scenes, we selectively removed long or short contours from the line drawings. In a category-matching task, participants performed significantly worse when long contours were removed than when short contours were removed. We conclude that global scene structure, which is preserved in line drawings, plays an integral part in representing scene categories.

Original languageEnglish (US)
Pages (from-to)9661-9666
Number of pages6
JournalProceedings of the National Academy of Sciences of the United States of America
Volume108
Issue number23
DOIs
StatePublished - Jun 7 2011

Fingerprint

Visual Cortex
Magnetic Resonance Imaging
Color

Keywords

  • Line art
  • Multivoxel pattern analysis
  • Neuroimaging
  • Scene perception
  • Visual processing

ASJC Scopus subject areas

  • General

Cite this

Simple line drawings suffice for functional MRI decoding of natural scene categories. / Walther, Dirk B.; Chai, Barry; Caddigan, Eamon; Beck, Diane M.; Fei-Fei, Li.

In: Proceedings of the National Academy of Sciences of the United States of America, Vol. 108, No. 23, 07.06.2011, p. 9661-9666.

Research output: Contribution to journalArticle

@article{d3895372223241c399faafc0c3b6285b,
title = "Simple line drawings suffice for functional MRI decoding of natural scene categories",
abstract = "Humans are remarkably efficient at categorizing natural scenes. In fact, scene categories can be decoded from functional MRI (fMRI) data throughout the ventral visual cortex, including the primary visual cortex, the parahippocampal place area (PPA), and the retrosplenial cortex (RSC). Here we ask whether, and where, we can still decode scene category if we reduce the scenes to mere lines. We collected fMRI data while participants viewed photographs and line drawings of beaches, city streets, forests, highways, mountains, and offices. Despite the marked difference in scene statistics, we were able to decode scene category from fMRI data for line drawings just as well as from activity for color photographs, in primary visual cortex through PPA and RSC. Even more remarkably, in PPA and RSC, error patterns for decoding from line drawings were very similar to those from color photographs. These data suggest that, in these regions, the information used to distinguish scene category is similar for line drawings and photographs. To determine the relative contributions of local and global structure to the human ability to categorize scenes, we selectively removed long or short contours from the line drawings. In a category-matching task, participants performed significantly worse when long contours were removed than when short contours were removed. We conclude that global scene structure, which is preserved in line drawings, plays an integral part in representing scene categories.",
keywords = "Line art, Multivoxel pattern analysis, Neuroimaging, Scene perception, Visual processing",
author = "Walther, {Dirk B.} and Barry Chai and Eamon Caddigan and Beck, {Diane M.} and Li Fei-Fei",
year = "2011",
month = "6",
day = "7",
doi = "10.1073/pnas.1015666108",
language = "English (US)",
volume = "108",
pages = "9661--9666",
journal = "Proceedings of the National Academy of Sciences of the United States of America",
issn = "0027-8424",
publisher = "National Academy of Sciences",
number = "23",

}

TY - JOUR

T1 - Simple line drawings suffice for functional MRI decoding of natural scene categories

AU - Walther, Dirk B.

AU - Chai, Barry

AU - Caddigan, Eamon

AU - Beck, Diane M.

AU - Fei-Fei, Li

PY - 2011/6/7

Y1 - 2011/6/7

N2 - Humans are remarkably efficient at categorizing natural scenes. In fact, scene categories can be decoded from functional MRI (fMRI) data throughout the ventral visual cortex, including the primary visual cortex, the parahippocampal place area (PPA), and the retrosplenial cortex (RSC). Here we ask whether, and where, we can still decode scene category if we reduce the scenes to mere lines. We collected fMRI data while participants viewed photographs and line drawings of beaches, city streets, forests, highways, mountains, and offices. Despite the marked difference in scene statistics, we were able to decode scene category from fMRI data for line drawings just as well as from activity for color photographs, in primary visual cortex through PPA and RSC. Even more remarkably, in PPA and RSC, error patterns for decoding from line drawings were very similar to those from color photographs. These data suggest that, in these regions, the information used to distinguish scene category is similar for line drawings and photographs. To determine the relative contributions of local and global structure to the human ability to categorize scenes, we selectively removed long or short contours from the line drawings. In a category-matching task, participants performed significantly worse when long contours were removed than when short contours were removed. We conclude that global scene structure, which is preserved in line drawings, plays an integral part in representing scene categories.

AB - Humans are remarkably efficient at categorizing natural scenes. In fact, scene categories can be decoded from functional MRI (fMRI) data throughout the ventral visual cortex, including the primary visual cortex, the parahippocampal place area (PPA), and the retrosplenial cortex (RSC). Here we ask whether, and where, we can still decode scene category if we reduce the scenes to mere lines. We collected fMRI data while participants viewed photographs and line drawings of beaches, city streets, forests, highways, mountains, and offices. Despite the marked difference in scene statistics, we were able to decode scene category from fMRI data for line drawings just as well as from activity for color photographs, in primary visual cortex through PPA and RSC. Even more remarkably, in PPA and RSC, error patterns for decoding from line drawings were very similar to those from color photographs. These data suggest that, in these regions, the information used to distinguish scene category is similar for line drawings and photographs. To determine the relative contributions of local and global structure to the human ability to categorize scenes, we selectively removed long or short contours from the line drawings. In a category-matching task, participants performed significantly worse when long contours were removed than when short contours were removed. We conclude that global scene structure, which is preserved in line drawings, plays an integral part in representing scene categories.

KW - Line art

KW - Multivoxel pattern analysis

KW - Neuroimaging

KW - Scene perception

KW - Visual processing

UR - http://www.scopus.com/inward/record.url?scp=79959329420&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79959329420&partnerID=8YFLogxK

U2 - 10.1073/pnas.1015666108

DO - 10.1073/pnas.1015666108

M3 - Article

C2 - 21593417

AN - SCOPUS:79959329420

VL - 108

SP - 9661

EP - 9666

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 23

ER -