Integration of partial information within and across modalities: Contributions to spoken and written sentence recognition

Kimberly G. Smith, Daniel Fogerty

Research output: Contribution to journalArticlepeer-review


Purpose: This study evaluated the extent to which partial spoken or written information facilitates sentence recognition under degraded unimodal and multimodal conditions. Method: Twenty young adults with typical hearing completed sentence recognition tasks in unimodal and multimodal conditions across 3 proportions of preservation. In the unimodal condition, performance was examined when only interrupted text or interrupted speech stimuli were available. In the multimodal condition, performance was examined when both interrupted text and interrupted speech stimuli were concurrently presented. Sentence recognition scores were obtained from simultaneous and delayed response conditions. Results: Significantly better performance was obtained for unimodal speech-only compared with text-only conditions across all proportions preserved. The multimodal condition revealed better performance when responses were delayed. During simultaneous responses, participants received equal benefit from speech information when the text was moderately and significantly degraded. The benefit from text in degraded auditory environments occurred only when speech was highly degraded. Conclusions: The speech signal, compared with text, is robust against degradation likely due to its continuous, versus discrete, features. Allowing time for offline linguistic processing is beneficial for the recognition of partial sensory information in unimodal and multimodal conditions. Despite the perceptual differences between the 2 modalities, the results highlight the utility of multimodal speech + text signals.

Original languageEnglish (US)
Pages (from-to)1805-1817
Number of pages13
JournalJournal of Speech, Language, and Hearing Research
Issue number6
StatePublished - Dec 2015
Externally publishedYes

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language
  • Speech and Hearing


Dive into the research topics of 'Integration of partial information within and across modalities: Contributions to spoken and written sentence recognition'. Together they form a unique fingerprint.

Cite this