Improving Bot Response Contradiction Detection via Utterance Rewriting

Di Jin, Sijia Liu, Yang Liu, Dilek Hakkani-Tur

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Though chatbots based on large neural models can often produce fluent responses in open domain conversations, one salient error type is contradiction or inconsistency with the preceding conversation turns. Previous work has treated contradiction detection in bot responses as a task similar to natural language inference, e.g., detect the contradiction between a pair of bot utterances. However, utterances in conversations may contain co-references or ellipsis, and using these utterances as is may not always be sufficient for identifying contradictions. This work aims to improve the contradiction detection via rewriting all bot utterances to restore antecedents and ellipsis. We curated a new dataset for utterance rewriting and built a rewriting model on it. We empirically demonstrate that this model can produce satisfactory rewrites to make bot utterances more complete. Furthermore, using rewritten utterances improves contradiction detection performance significantly, e.g., the AUPR and joint accuracy scores (detecting contradiction along with evidence) increase by 6.5% and 4.5% (absolute increase), respectively.

Original languageEnglish (US)
Title of host publicationSIGDIAL 2022 - 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference
PublisherAssociation for Computational Linguistics (ACL)
Pages605-614
Number of pages10
ISBN (Electronic)9781955917667
StatePublished - 2022
Externally publishedYes
Event23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2022 - Edinburgh, United Kingdom
Duration: Sep 7 2022Sep 9 2022

Publication series

NameSIGDIAL 2022 - 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference

Conference

Conference23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2022
Country/TerritoryUnited Kingdom
CityEdinburgh
Period9/7/229/9/22

ASJC Scopus subject areas

  • Modeling and Simulation
  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction

Cite this