Trajectory-based Fisher kernel representation for action recognition in videos

Indriyati Atmosukarto, Bernard Ghanem, Narendra Ahuja

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Action recognition is an important computer vision problem that has many applications including video indexing and retrieval, event detection, and video summarization. In this paper, we propose to apply the Fisher kernel paradigm to action recognition. The Fisher kernel framework combines the strengths of generative and discriminative models. In this approach, given the trajectories extracted from a video and a generative Gaussian Mixture Model (GMM), we use the Fisher Kernel method to describe how much the GMM parameters are modified to best fit the video trajectories. We experiment in using the Fisher Kernel vector to create the video representation and to train an SVM classifier. We further extend our framework to select the most discriminative trajectories using a novel MIL-KNN framework. We compare the performance of our approach to the current state-of-the-art bag-of-features (BOF) approach on two benchmark datasets. Experimental results show that our proposed approach outperforms the state-of-the-art method [8] and that the selected discriminative trajectories are descriptive of the action class.

Original languageEnglish (US)
Title of host publicationICPR 2012 - 21st International Conference on Pattern Recognition
Pages3333-3336
Number of pages4
StatePublished - Dec 1 2012
Event21st International Conference on Pattern Recognition, ICPR 2012 - Tsukuba, Japan
Duration: Nov 11 2012Nov 15 2012

Publication series

NameProceedings - International Conference on Pattern Recognition
ISSN (Print)1051-4651

Other

Other21st International Conference on Pattern Recognition, ICPR 2012
CountryJapan
CityTsukuba
Period11/11/1211/15/12

Fingerprint

Trajectories
Computer vision
Classifiers
Experiments

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Cite this

Atmosukarto, I., Ghanem, B., & Ahuja, N. (2012). Trajectory-based Fisher kernel representation for action recognition in videos. In ICPR 2012 - 21st International Conference on Pattern Recognition (pp. 3333-3336). [6460878] (Proceedings - International Conference on Pattern Recognition).

Trajectory-based Fisher kernel representation for action recognition in videos. / Atmosukarto, Indriyati; Ghanem, Bernard; Ahuja, Narendra.

ICPR 2012 - 21st International Conference on Pattern Recognition. 2012. p. 3333-3336 6460878 (Proceedings - International Conference on Pattern Recognition).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Atmosukarto, I, Ghanem, B & Ahuja, N 2012, Trajectory-based Fisher kernel representation for action recognition in videos. in ICPR 2012 - 21st International Conference on Pattern Recognition., 6460878, Proceedings - International Conference on Pattern Recognition, pp. 3333-3336, 21st International Conference on Pattern Recognition, ICPR 2012, Tsukuba, Japan, 11/11/12.
Atmosukarto I, Ghanem B, Ahuja N. Trajectory-based Fisher kernel representation for action recognition in videos. In ICPR 2012 - 21st International Conference on Pattern Recognition. 2012. p. 3333-3336. 6460878. (Proceedings - International Conference on Pattern Recognition).
Atmosukarto, Indriyati ; Ghanem, Bernard ; Ahuja, Narendra. / Trajectory-based Fisher kernel representation for action recognition in videos. ICPR 2012 - 21st International Conference on Pattern Recognition. 2012. pp. 3333-3336 (Proceedings - International Conference on Pattern Recognition).
@inproceedings{90468ef41d39451aa2bfcfa335439f49,
title = "Trajectory-based Fisher kernel representation for action recognition in videos",
abstract = "Action recognition is an important computer vision problem that has many applications including video indexing and retrieval, event detection, and video summarization. In this paper, we propose to apply the Fisher kernel paradigm to action recognition. The Fisher kernel framework combines the strengths of generative and discriminative models. In this approach, given the trajectories extracted from a video and a generative Gaussian Mixture Model (GMM), we use the Fisher Kernel method to describe how much the GMM parameters are modified to best fit the video trajectories. We experiment in using the Fisher Kernel vector to create the video representation and to train an SVM classifier. We further extend our framework to select the most discriminative trajectories using a novel MIL-KNN framework. We compare the performance of our approach to the current state-of-the-art bag-of-features (BOF) approach on two benchmark datasets. Experimental results show that our proposed approach outperforms the state-of-the-art method [8] and that the selected discriminative trajectories are descriptive of the action class.",
author = "Indriyati Atmosukarto and Bernard Ghanem and Narendra Ahuja",
year = "2012",
month = "12",
day = "1",
language = "English (US)",
isbn = "9784990644109",
series = "Proceedings - International Conference on Pattern Recognition",
pages = "3333--3336",
booktitle = "ICPR 2012 - 21st International Conference on Pattern Recognition",

}

TY - GEN

T1 - Trajectory-based Fisher kernel representation for action recognition in videos

AU - Atmosukarto, Indriyati

AU - Ghanem, Bernard

AU - Ahuja, Narendra

PY - 2012/12/1

Y1 - 2012/12/1

N2 - Action recognition is an important computer vision problem that has many applications including video indexing and retrieval, event detection, and video summarization. In this paper, we propose to apply the Fisher kernel paradigm to action recognition. The Fisher kernel framework combines the strengths of generative and discriminative models. In this approach, given the trajectories extracted from a video and a generative Gaussian Mixture Model (GMM), we use the Fisher Kernel method to describe how much the GMM parameters are modified to best fit the video trajectories. We experiment in using the Fisher Kernel vector to create the video representation and to train an SVM classifier. We further extend our framework to select the most discriminative trajectories using a novel MIL-KNN framework. We compare the performance of our approach to the current state-of-the-art bag-of-features (BOF) approach on two benchmark datasets. Experimental results show that our proposed approach outperforms the state-of-the-art method [8] and that the selected discriminative trajectories are descriptive of the action class.

AB - Action recognition is an important computer vision problem that has many applications including video indexing and retrieval, event detection, and video summarization. In this paper, we propose to apply the Fisher kernel paradigm to action recognition. The Fisher kernel framework combines the strengths of generative and discriminative models. In this approach, given the trajectories extracted from a video and a generative Gaussian Mixture Model (GMM), we use the Fisher Kernel method to describe how much the GMM parameters are modified to best fit the video trajectories. We experiment in using the Fisher Kernel vector to create the video representation and to train an SVM classifier. We further extend our framework to select the most discriminative trajectories using a novel MIL-KNN framework. We compare the performance of our approach to the current state-of-the-art bag-of-features (BOF) approach on two benchmark datasets. Experimental results show that our proposed approach outperforms the state-of-the-art method [8] and that the selected discriminative trajectories are descriptive of the action class.

UR - http://www.scopus.com/inward/record.url?scp=84874577192&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84874577192&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84874577192

SN - 9784990644109

T3 - Proceedings - International Conference on Pattern Recognition

SP - 3333

EP - 3336

BT - ICPR 2012 - 21st International Conference on Pattern Recognition

ER -