Mirror Hearts: Exploring the (Mis-)Alignment between AI-Recognized and Self-Reported Emotions

Si Chen, Haocong Cheng, Jason Situ, Yun Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Previous research has found that learners' reflections on their own emotions can improve their learning experience, and AI-based technologies can be used to automatically recognize learners' emotions. We conducted user studies involving 32 participants to investigate the relationship between AI-recognized emotions and their self-reported emotions using emojis and text comments. We found that, even though AI recognized a similar amount of positive-high-arousal and negative-low-arousal emotions, participants self-reported more positive-high-arousal emojis. Participants explained the causality and temporality of the self-reported emojis using text comments. Our findings suggest ways of using AI to capture learners' emotions and support their reflections.

Original languageEnglish (US)
Title of host publicationCHI 2023 - Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450394222
DOIs
StatePublished - Apr 19 2023
Event2023 CHI Conference on Human Factors in Computing Systems, CHI 2023 - Hamburg, Germany
Duration: Apr 23 2023Apr 28 2023

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2023 CHI Conference on Human Factors in Computing Systems, CHI 2023
Country/TerritoryGermany
CityHamburg
Period4/23/234/28/23

Keywords

  • Emoji
  • Facial Recognition
  • Reflection
  • Self-Regulated Learning

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Mirror Hearts: Exploring the (Mis-)Alignment between AI-Recognized and Self-Reported Emotions'. Together they form a unique fingerprint.

Cite this