TY - GEN
T1 - 'How Robust R U?'
T2 - 2021 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2021
AU - Kim, Seokhwan
AU - Liu, Yang
AU - Jin, Di
AU - Papangelis, Alexandros
AU - Gopalakrishnan, Karthik
AU - Hedayatnia, Behnam
AU - Hakkani-Tur, Dilek
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Most prior work in dialogue modeling has been on written conversations mostly because of existing data sets. However, written dialogues are not sufficient to fully capture the nature of spoken conversations as well as the potential speech recognition errors in practical spoken dialogue systems. This work presents a new benchmark on spoken task-oriented conversations, which is intended to study multi-domain dialogue state tracking and knowledge-grounded dialogue modeling. We report that the existing state-of-the-art models trained on written conversations are not performing well on our spoken data, as expected. Furthermore, we observe improvements in task performances when leveraging $n$-best speech recognition hypotheses such as by combining predictions based on individual hypotheses. Our data set enables speech-based benchmarking of task-oriented dialogue systems.
AB - Most prior work in dialogue modeling has been on written conversations mostly because of existing data sets. However, written dialogues are not sufficient to fully capture the nature of spoken conversations as well as the potential speech recognition errors in practical spoken dialogue systems. This work presents a new benchmark on spoken task-oriented conversations, which is intended to study multi-domain dialogue state tracking and knowledge-grounded dialogue modeling. We report that the existing state-of-the-art models trained on written conversations are not performing well on our spoken data, as expected. Furthermore, we observe improvements in task performances when leveraging $n$-best speech recognition hypotheses such as by combining predictions based on individual hypotheses. Our data set enables speech-based benchmarking of task-oriented dialogue systems.
KW - dialogue state tracking
KW - knowledge-grounded dialogue generation
KW - spoken dialogue systems
UR - https://www.scopus.com/pages/publications/85126772366
UR - https://www.scopus.com/pages/publications/85126772366#tab=citedBy
U2 - 10.1109/ASRU51503.2021.9688274
DO - 10.1109/ASRU51503.2021.9688274
M3 - Conference contribution
AN - SCOPUS:85126772366
T3 - 2021 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2021 - Proceedings
SP - 1147
EP - 1154
BT - 2021 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2021 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 13 December 2021 through 17 December 2021
ER -