Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences

Charissa R. Lansing, George W. McConkie

Research output: Contribution to journalArticle

Abstract

In this study, we investigated where people look on talkers' faces as they try to understand what is being said. Sixteen young adults with normal hearing and demonstrated average speechreading proficiency were evaluated under two modality presentation conditions: vision only versus vision plus low-intensity sound. They were scored for the number of words correctly identified from 80 unconnected sentences spoken by two talkers. The results showed two competing tendencies: an eye primacy effect that draws the gaze to the talker's eyes during silence and an information source attraction effect that draws the gaze to the talker's mouth during speech periods. Dynamic shifts occur between eyes and mouth prior to speech onset and following the offset of speech, and saccades tend to be suppressed during speech periods. The degree to which the gaze is drawn to the mouth during speech and the degree to which saccadic activity is suppressed depend on the difficulty of the speech identification task. Under the most difficult modality presentation condition, vision only, accuracy was related to average sentence difficulty and individual proficiency in visual speech perception, but not to the proportion of gaze time directed toward the talker's mouth or toward other parts of the talker's face.

Original languageEnglish (US)
Pages (from-to)536-552
Number of pages17
JournalPerception and Psychophysics
Volume65
Issue number4
DOIs
StatePublished - May 2003

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Sensory Systems
  • Psychology(all)

Fingerprint Dive into the research topics of 'Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences'. Together they form a unique fingerprint.

  • Cite this