Reversing the Hands of Time: Changing the Mapping From Seeing to Saying

Stefanie E. Kuchinsky, Kathryn Bock, David E Irwin

Research output: Contribution to journalArticle

Abstract

To describe a scene, speakers must map visual information to a linguistic plan. Eye movements capture features of this linkage in a tendency for speakers to fixate referents just before they are mentioned. The current experiment examined whether and how this pattern changes when speakers create atypical mappings. Eye movements were monitored as participants told the time from analog clocks. Half of the participants did this in the usual manner. For the other participants, the denotations of the clock hands were reversed, making the big hand the hour and the little hand the minute. Eye movements revealed that it was not the visual features or configuration of the hands that determined gaze patterns, but rather top-down control from upcoming referring expressions. Differences in eye-voice spans further suggested a process in which scene elements are relationally structured before a linguistic plan is executed. This provides evidence for structural rather than lexical incrementality in planning and supports a " seeing-for-saying" hypothesis in which the visual system is harnessed to the linguistic demands of an upcoming utterance.

Original languageEnglish (US)
Pages (from-to)748-756
Number of pages9
JournalJournal of Experimental Psychology: Learning Memory and Cognition
Volume37
Issue number3
DOIs
StatePublished - May 1 2011

    Fingerprint

Keywords

  • Eyetracking
  • Language production
  • Time telling

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Language and Linguistics
  • Linguistics and Language

Cite this