Prosody dependent speech recognition on radio news corpus of American English

Ken Chen, Mark Hasegawa-Johnson, Aaron Cohen, Sarah Borys, Sung Suk Kim, Jennifer Cole, Jeung Yoon Choi

Research output: Contribution to journalArticle

Abstract

Does prosody help word recognition? This paper proposes a novel probabilistic framework in which word and phoneme are dependent on prosody in a way that reduces word error rates (WER) relative to a prosody-independent recognizer with comparable parameter count. In the proposed prosody-dependent speech recognizer, word and phoneme models are conditioned on two important prosodic variables: the intonational phrase boundary and the pitch accent. An information-theoretic analysis is provided to show that prosody dependent acoustic and language modeling can increase the mutual information between the true word hypothesis and the acoustic observation by exciting the interaction between prosody dependent acoustic model and prosody dependent language model. Empirically, results indicate that the influence of these prosodic variables on allophonic models are mainly restricted to a small subset of distributions: the duration PDFs (modeled using an explicit duration hidden Markov model or EDHMM) and the acoustic-prosodic observation PDFs (normalized pitch frequency). Influence of prosody on cepstral features is limited to a subset of phonemes: for example, vowels may be influenced by both accent and phrase position, but phrase-initial and phrase-final consonants are independent of accent. Leveraging these results, effective prosody dependent allophonic models are built with minimal increase in parameter count. These prosody dependent speech recognizers are able to reduce word error rates by up to 11 % relative to prosody independent recognizers with comparable parameter count, in experiments based on the prosodically-transcribed Boston Radio News corpus.

Original languageEnglish (US)
Pages (from-to)232-244
Number of pages13
JournalIEEE Transactions on Audio, Speech and Language Processing
Volume14
Issue number1
DOIs
StatePublished - Jan 1 2006

Keywords

  • ANN
  • Acoustic model
  • Duration
  • HMM
  • Mutual information
  • Pitch
  • Prosody
  • ToBI
  • Word error rate

ASJC Scopus subject areas

  • Acoustics and Ultrasonics
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Prosody dependent speech recognition on radio news corpus of American English'. Together they form a unique fingerprint.

  • Cite this