Human-Object Interactions Are More than the Sum of Their Parts

Christopher Baldassano, Diane M. Beck, Li Fei-Fei

Research output: Contribution to journalArticle

Abstract

Understanding human-object interactions is critical for extracting meaning from everyday visual scenes and requires integrating complex relationships between human pose and object identity into a new percept. To understand how the brain builds these representations, we conducted 2 fMRI experiments in which subjects viewed humans interacting with objects, noninteracting human-object pairs, and isolated humans and objects. A number of visual regions process features of human-object interactions, including object identity information in the lateral occipital complex (LOC) and parahippocampal place area (PPA), and human pose information in the extrastriate body area (EBA) and posterior superior temporal sulcus (pSTS). Representations of human-object interactions in some regions, such as the posterior PPA (retinotopic maps PHC1 and PHC2) are well predicted by a simple linear combination of the response to object and pose information. Other regions, however, especially pSTS, exhibit representations for human-object interaction categories that are not predicted by their individual components, indicating that they encode human-object interactions as more than the sum of their parts. These results reveal the distributed networks underlying the emergent representation of human-object interactions necessary for social perception.

Original languageEnglish (US)
Pages (from-to)2276-2288
Number of pages13
JournalCerebral cortex (New York, N.Y. : 1991)
Volume27
Issue number3
DOIs
StatePublished - Mar 1 2017

Fingerprint

Temporal Lobe
Social Perception
Magnetic Resonance Imaging
Brain

Keywords

  • MVPA
  • action perception
  • cross-decoding
  • fMRI
  • scene perception

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Cellular and Molecular Neuroscience

Cite this

Human-Object Interactions Are More than the Sum of Their Parts. / Baldassano, Christopher; Beck, Diane M.; Fei-Fei, Li.

In: Cerebral cortex (New York, N.Y. : 1991), Vol. 27, No. 3, 01.03.2017, p. 2276-2288.

Research output: Contribution to journalArticle

Baldassano, Christopher ; Beck, Diane M. ; Fei-Fei, Li. / Human-Object Interactions Are More than the Sum of Their Parts. In: Cerebral cortex (New York, N.Y. : 1991). 2017 ; Vol. 27, No. 3. pp. 2276-2288.
@article{379ed7afb82c45ea86fda58190f8a455,
title = "Human-Object Interactions Are More than the Sum of Their Parts",
abstract = "Understanding human-object interactions is critical for extracting meaning from everyday visual scenes and requires integrating complex relationships between human pose and object identity into a new percept. To understand how the brain builds these representations, we conducted 2 fMRI experiments in which subjects viewed humans interacting with objects, noninteracting human-object pairs, and isolated humans and objects. A number of visual regions process features of human-object interactions, including object identity information in the lateral occipital complex (LOC) and parahippocampal place area (PPA), and human pose information in the extrastriate body area (EBA) and posterior superior temporal sulcus (pSTS). Representations of human-object interactions in some regions, such as the posterior PPA (retinotopic maps PHC1 and PHC2) are well predicted by a simple linear combination of the response to object and pose information. Other regions, however, especially pSTS, exhibit representations for human-object interaction categories that are not predicted by their individual components, indicating that they encode human-object interactions as more than the sum of their parts. These results reveal the distributed networks underlying the emergent representation of human-object interactions necessary for social perception.",
keywords = "MVPA, action perception, cross-decoding, fMRI, scene perception",
author = "Christopher Baldassano and Beck, {Diane M.} and Li Fei-Fei",
year = "2017",
month = "3",
day = "1",
doi = "10.1093/cercor/bhw077",
language = "English (US)",
volume = "27",
pages = "2276--2288",
journal = "Cerebral Cortex",
issn = "1047-3211",
publisher = "Oxford University Press",
number = "3",

}

TY - JOUR

T1 - Human-Object Interactions Are More than the Sum of Their Parts

AU - Baldassano, Christopher

AU - Beck, Diane M.

AU - Fei-Fei, Li

PY - 2017/3/1

Y1 - 2017/3/1

N2 - Understanding human-object interactions is critical for extracting meaning from everyday visual scenes and requires integrating complex relationships between human pose and object identity into a new percept. To understand how the brain builds these representations, we conducted 2 fMRI experiments in which subjects viewed humans interacting with objects, noninteracting human-object pairs, and isolated humans and objects. A number of visual regions process features of human-object interactions, including object identity information in the lateral occipital complex (LOC) and parahippocampal place area (PPA), and human pose information in the extrastriate body area (EBA) and posterior superior temporal sulcus (pSTS). Representations of human-object interactions in some regions, such as the posterior PPA (retinotopic maps PHC1 and PHC2) are well predicted by a simple linear combination of the response to object and pose information. Other regions, however, especially pSTS, exhibit representations for human-object interaction categories that are not predicted by their individual components, indicating that they encode human-object interactions as more than the sum of their parts. These results reveal the distributed networks underlying the emergent representation of human-object interactions necessary for social perception.

AB - Understanding human-object interactions is critical for extracting meaning from everyday visual scenes and requires integrating complex relationships between human pose and object identity into a new percept. To understand how the brain builds these representations, we conducted 2 fMRI experiments in which subjects viewed humans interacting with objects, noninteracting human-object pairs, and isolated humans and objects. A number of visual regions process features of human-object interactions, including object identity information in the lateral occipital complex (LOC) and parahippocampal place area (PPA), and human pose information in the extrastriate body area (EBA) and posterior superior temporal sulcus (pSTS). Representations of human-object interactions in some regions, such as the posterior PPA (retinotopic maps PHC1 and PHC2) are well predicted by a simple linear combination of the response to object and pose information. Other regions, however, especially pSTS, exhibit representations for human-object interaction categories that are not predicted by their individual components, indicating that they encode human-object interactions as more than the sum of their parts. These results reveal the distributed networks underlying the emergent representation of human-object interactions necessary for social perception.

KW - MVPA

KW - action perception

KW - cross-decoding

KW - fMRI

KW - scene perception

UR - http://www.scopus.com/inward/record.url?scp=85021389557&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021389557&partnerID=8YFLogxK

U2 - 10.1093/cercor/bhw077

DO - 10.1093/cercor/bhw077

M3 - Article

C2 - 27073216

AN - SCOPUS:85021389557

VL - 27

SP - 2276

EP - 2288

JO - Cerebral Cortex

JF - Cerebral Cortex

SN - 1047-3211

IS - 3

ER -