Face forward: Detecting mind wandering from video during narrative film comprehension

Angela Stewart, Nigel Bosch, Huili Chen, Patrick Donnelly, Sidney D’Mello

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Attention is key to effective learning, but mind wandering, a phenomenon in which attention shifts from task-related processing to task-unrelated thoughts, is pervasive across learning tasks. Therefore, intelligent learning environments should benefit from mechanisms to detect and respond to attentional lapses, such as mind wandering. As a step in this direction, we report the development and validation of the first student-independent facial feature-based mind wandering detector. We collected training data in a lab study where participants self-reported when they caught themselves mind wandering over the course of completing a 32.5 min narrative film comprehension task. We used computer vision techniques to extract facial features and bodily movements from videos. Using supervised learning methods, we were able to detect a mind wandering with an F1 score of.390, which reflected a 31% improvement over a chance model. We discuss how our mind wandering detector can be used to adapt the learning experience, particularly for online learning contexts.

Original languageEnglish (US)
Title of host publicationArtificial Intelligence in Education - 18th International Conference, AIED 2017, Proceedings
EditorsElisabeth Andre, Xiangen Hu, Ma. Mercedes T. Rodrigo, Benedict du Boulay, Ryan Baker
PublisherSpringer
Pages359-370
Number of pages12
ISBN (Print)9783319614243
DOIs
StatePublished - 2017
Event18th International Conference on Artificial Intelligence in Education, AIED 2017 - Wuhan, China
Duration: Jun 28 2017Jul 1 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10331 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other18th International Conference on Artificial Intelligence in Education, AIED 2017
Country/TerritoryChina
CityWuhan
Period6/28/177/1/17

Keywords

  • Attention aware interfaces
  • Mind wandering

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Face forward: Detecting mind wandering from video during narrative film comprehension'. Together they form a unique fingerprint.

Cite this