To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions

Hyo Jin Do, Ha Kyung Kong, Pooja Tetali, Jaewook Lee, Brian P. Bailey

Research output: Contribution to journalArticlepeer-review

Abstract

Conversational agents (CAs) can analyze online conversations using natural language techniques and effectively facilitate group discussions by sending supervisory messages. However, if a CA makes imperfect interventions, users may stop trusting the CA and discontinue using it. In this study, we demonstrate how inaccurate interventions of a CA and a conversational repair strategy can influence user acceptance of the CA, members' participation in the discussion, perceived discussion experience between the members, and group performance. We built a CA that encourages the participation of members with low contributions in an online chat discussion in which a small group (3-6 members) performs a decision-making task. Two types of errors can occur when detecting under-contributing members: 1) false-positive (FP) errors happen when the CA falsely identifies a member as under-contributing and 2) false-negative (FN) errors occur when the CA misses detecting an under-contributing member. We designed a conversational repair strategy that gives users a chance to contest the detection results and the agent sends a correctional message if an error is detected. Through an online study with 175 participants, we found that participants who received FN error messages reported higher acceptance of the CA and better discussion experience, but participated less compared to those who received FP error messages. The conversational repair strategy moderated the effect of errors such as improving the perceived discussion experience of participants who received FP error messages. Based on our findings, we offer design implications for which model should be selected by practitioners between high precision (i.e., fewer FP errors) and high recall (i.e., fewer FN errors) models depending on the desired effects. When frequent FP errors are expected, we suggest using the conversational repair strategy to improve the perceived discussion experience.

Original languageEnglish (US)
Article number99
JournalProceedings of the ACM on Human-Computer Interaction
Volume7
Issue number1 CSCW
DOIs
StatePublished - Apr 16 2023

Keywords

  • collaborative task
  • conversational agent
  • group discussion
  • participation
  • user acceptance

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Human-Computer Interaction
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions'. Together they form a unique fingerprint.

Cite this