The static multimodal dyadic behavior dataset for engagement prediction

P. Daphne Tsatsoulis, Paige Kordas, Michael Marshall, David Alexander Forsyth, Agata Rozga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The Rapid-Attention, Back and Forth, and Communication (Rapid ABC) assessment is a semi-structured play interaction during which an examiner engages a child in five activities intended to elicit social-communication behaviors and turn taking. The examiner scores the frequency and quality of the child’s social behavior in each activity, generating a total score that reflects the child’s social engagement with her during the assessment. The standard Rapid ABC dataset contains a daunting amount of detail. We have produced a static version that captures the action-reaction dynamic of the assessment as frames. We have conducted a user study on our dataset to see if subjects can predict the engagement of a child in the video.We presented subjects both frames from our staticMMDB dataset and the full video of the original MMDB dataset and found little difference in their performance. In this paper we show that computer vision methods can predict children’s engagement. We automatically identify the ease-of-engagement of a child and provide evaluation baselines for the task.

Original languageEnglish (US)
Title of host publicationComputer Vision – ECCV 2016 Workshops, Proceedings
EditorsGang Hua, Herve Jegou
PublisherSpringer-Verlag
Pages386-399
Number of pages14
ISBN (Print)9783319494081
DOIs
StatePublished - Jan 1 2016
Event14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands
Duration: Oct 11 2016Oct 14 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9915 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other14th European Conference on Computer Vision, ECCV 2016
CountryNetherlands
CityAmsterdam
Period10/11/1610/14/16

Fingerprint

Prediction
Communication
Computer vision
Predict
Social Behavior
User Studies
Computer Vision
Baseline
Children
Engagement
Evaluation
Interaction
Standards

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Tsatsoulis, P. D., Kordas, P., Marshall, M., Forsyth, D. A., & Rozga, A. (2016). The static multimodal dyadic behavior dataset for engagement prediction. In G. Hua, & H. Jegou (Eds.), Computer Vision – ECCV 2016 Workshops, Proceedings (pp. 386-399). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9915 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-319-49409-8_31

The static multimodal dyadic behavior dataset for engagement prediction. / Tsatsoulis, P. Daphne; Kordas, Paige; Marshall, Michael; Forsyth, David Alexander; Rozga, Agata.

Computer Vision – ECCV 2016 Workshops, Proceedings. ed. / Gang Hua; Herve Jegou. Springer-Verlag, 2016. p. 386-399 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9915 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tsatsoulis, PD, Kordas, P, Marshall, M, Forsyth, DA & Rozga, A 2016, The static multimodal dyadic behavior dataset for engagement prediction. in G Hua & H Jegou (eds), Computer Vision – ECCV 2016 Workshops, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9915 LNCS, Springer-Verlag, pp. 386-399, 14th European Conference on Computer Vision, ECCV 2016, Amsterdam, Netherlands, 10/11/16. https://doi.org/10.1007/978-3-319-49409-8_31
Tsatsoulis PD, Kordas P, Marshall M, Forsyth DA, Rozga A. The static multimodal dyadic behavior dataset for engagement prediction. In Hua G, Jegou H, editors, Computer Vision – ECCV 2016 Workshops, Proceedings. Springer-Verlag. 2016. p. 386-399. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-49409-8_31
Tsatsoulis, P. Daphne ; Kordas, Paige ; Marshall, Michael ; Forsyth, David Alexander ; Rozga, Agata. / The static multimodal dyadic behavior dataset for engagement prediction. Computer Vision – ECCV 2016 Workshops, Proceedings. editor / Gang Hua ; Herve Jegou. Springer-Verlag, 2016. pp. 386-399 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{11541d30e1364933a2f88589f4dbc86e,
title = "The static multimodal dyadic behavior dataset for engagement prediction",
abstract = "The Rapid-Attention, Back and Forth, and Communication (Rapid ABC) assessment is a semi-structured play interaction during which an examiner engages a child in five activities intended to elicit social-communication behaviors and turn taking. The examiner scores the frequency and quality of the child’s social behavior in each activity, generating a total score that reflects the child’s social engagement with her during the assessment. The standard Rapid ABC dataset contains a daunting amount of detail. We have produced a static version that captures the action-reaction dynamic of the assessment as frames. We have conducted a user study on our dataset to see if subjects can predict the engagement of a child in the video.We presented subjects both frames from our staticMMDB dataset and the full video of the original MMDB dataset and found little difference in their performance. In this paper we show that computer vision methods can predict children’s engagement. We automatically identify the ease-of-engagement of a child and provide evaluation baselines for the task.",
author = "Tsatsoulis, {P. Daphne} and Paige Kordas and Michael Marshall and Forsyth, {David Alexander} and Agata Rozga",
year = "2016",
month = "1",
day = "1",
doi = "10.1007/978-3-319-49409-8_31",
language = "English (US)",
isbn = "9783319494081",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer-Verlag",
pages = "386--399",
editor = "Gang Hua and Herve Jegou",
booktitle = "Computer Vision – ECCV 2016 Workshops, Proceedings",

}

TY - GEN

T1 - The static multimodal dyadic behavior dataset for engagement prediction

AU - Tsatsoulis, P. Daphne

AU - Kordas, Paige

AU - Marshall, Michael

AU - Forsyth, David Alexander

AU - Rozga, Agata

PY - 2016/1/1

Y1 - 2016/1/1

N2 - The Rapid-Attention, Back and Forth, and Communication (Rapid ABC) assessment is a semi-structured play interaction during which an examiner engages a child in five activities intended to elicit social-communication behaviors and turn taking. The examiner scores the frequency and quality of the child’s social behavior in each activity, generating a total score that reflects the child’s social engagement with her during the assessment. The standard Rapid ABC dataset contains a daunting amount of detail. We have produced a static version that captures the action-reaction dynamic of the assessment as frames. We have conducted a user study on our dataset to see if subjects can predict the engagement of a child in the video.We presented subjects both frames from our staticMMDB dataset and the full video of the original MMDB dataset and found little difference in their performance. In this paper we show that computer vision methods can predict children’s engagement. We automatically identify the ease-of-engagement of a child and provide evaluation baselines for the task.

AB - The Rapid-Attention, Back and Forth, and Communication (Rapid ABC) assessment is a semi-structured play interaction during which an examiner engages a child in five activities intended to elicit social-communication behaviors and turn taking. The examiner scores the frequency and quality of the child’s social behavior in each activity, generating a total score that reflects the child’s social engagement with her during the assessment. The standard Rapid ABC dataset contains a daunting amount of detail. We have produced a static version that captures the action-reaction dynamic of the assessment as frames. We have conducted a user study on our dataset to see if subjects can predict the engagement of a child in the video.We presented subjects both frames from our staticMMDB dataset and the full video of the original MMDB dataset and found little difference in their performance. In this paper we show that computer vision methods can predict children’s engagement. We automatically identify the ease-of-engagement of a child and provide evaluation baselines for the task.

UR - http://www.scopus.com/inward/record.url?scp=85006014536&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85006014536&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-49409-8_31

DO - 10.1007/978-3-319-49409-8_31

M3 - Conference contribution

AN - SCOPUS:85006014536

SN - 9783319494081

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 386

EP - 399

BT - Computer Vision – ECCV 2016 Workshops, Proceedings

A2 - Hua, Gang

A2 - Jegou, Herve

PB - Springer-Verlag

ER -