How deep neural networks can improve emotion recognition on video data

Pooya Khorrami, Tom Le Paine, Kevin Brady, Charlie Dagli, Thomas S. Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider the task of dimensional emotion recognition on video data using deep learning. While several previous methods have shown the benefits of training temporal neural network models such as recurrent neural networks (RNNs) on hand-crafted features, few works have considered combining convolutional neural networks (CNNs) with RNNs. In this work, we present a system that performs emotion recognition on video data using both CNNs and RNNs, and we also analyze how much each neural network component contributes to the system's overall performance. We present our findings on videos from the Audio/Visual+Emotion Challenge (AV+EC2015). In our experiments, we analyze the effects of several hyperparameters on overall performance while also achieving superior performance to the baseline and other competing methods.

Original languageEnglish (US)
Title of host publication2016 IEEE International Conference on Image Processing, ICIP 2016 - Proceedings
PublisherIEEE Computer Society
Pages619-623
Number of pages5
ISBN (Electronic)9781467399616
DOIs
StatePublished - Aug 3 2016
Event23rd IEEE International Conference on Image Processing, ICIP 2016 - Phoenix, United States
Duration: Sep 25 2016Sep 28 2016

Publication series

NameProceedings - International Conference on Image Processing, ICIP
Volume2016-August
ISSN (Print)1522-4880

Other

Other23rd IEEE International Conference on Image Processing, ICIP 2016
CountryUnited States
CityPhoenix
Period9/25/169/28/16

Keywords

  • Convolutional Neural Networks
  • Deep Learning
  • Emotion Recognition
  • Recurrent Neural Networks
  • Video Processing

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing

Fingerprint Dive into the research topics of 'How deep neural networks can improve emotion recognition on video data'. Together they form a unique fingerprint.

  • Cite this

    Khorrami, P., Le Paine, T., Brady, K., Dagli, C., & Huang, T. S. (2016). How deep neural networks can improve emotion recognition on video data. In 2016 IEEE International Conference on Image Processing, ICIP 2016 - Proceedings (pp. 619-623). [7532431] (Proceedings - International Conference on Image Processing, ICIP; Vol. 2016-August). IEEE Computer Society. https://doi.org/10.1109/ICIP.2016.7532431