Improving the robustness of prosody dependent language modeling based on prosody syntax dependence

This paper presents a novel approach that improves the robustness of prosody dependent language modeling by lever-aging the dependence between prosody and syntax. A prosody dependent language model describes the joint probability distribution of concurrent word and prosody sequences and can be used to provide prior language constraints in a prosody dependent speech recognizer. Robust Maximum Likelihood (ML) estimation of prosody dependent n-gram language models requires a large amount of prosodically transcribed data. In this paper, we show that the prosody-syntax dependence can be utilized to diminish the data sparseness introduced by prosody dependent modeling. Experiments on Radio News Corpus show that the prosody dependent language model estimated using our approach reduces the joint perplexity by up to 34% as compared with the standard ML-estimated prosody dependent language model; the word perplexity can be reduced by up to 84% as compared with the standard ML-estimated prosody independent language model. In recognition experiments, the language model estimated by our approach create an improvement of 1% in word recognition accuracy, 0.7% in accent recognition accuracy and 1.5% in intonational phrase boundary (IPB) recognition accuracy over a baseline prosody dependent language model.