Automatic video annotation using multimodal Dirichlet process mixture model

Atulya Velivelli, Thomas S Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper we infer a multimodal Dirichlet process mixture model from video data, the mixture components in this model follow a Gaussian-Multinomial distribution. The multimodal Dirichlet process mixture model clusters freely available multimodal data in videos i.e., the combination of visual track and the corresponding keywords extracted from speech transcripts obtained from the audio track of videos, using the parameters of the model we build a predictive model that can output keyword annotations given video shots. In the multimodal Dirichlet process mixture model the keywords follow a multinomial distribution while the features used to represent the video shot follow a Gaussian distribution. We infer the multimodal Dirichlet process mixture model by collecting samples from the corresponding markov chain using a Blocked Gibbs sampling algorithm, and use the inferred parameters to predict video shot annotations that can be used to perform text based retrieval of shots. We compare the performance of our proposed model with other baseline models that use predicted annotations for retrieval.

Original languageEnglish (US)
Title of host publicationProceedings of 2008 IEEE International Conference on Networking, Sensing and Control, ICNSC
Pages1366-1371
Number of pages6
DOIs
StatePublished - Aug 18 2008
Event2008 IEEE International Conference on Networking, Sensing and Control, ICNSC - Sanya, China
Duration: Apr 6 2008Apr 8 2008

Publication series

NameProceedings of 2008 IEEE International Conference on Networking, Sensing and Control, ICNSC

Other

Other2008 IEEE International Conference on Networking, Sensing and Control, ICNSC
CountryChina
CitySanya
Period4/6/084/8/08

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Control and Systems Engineering

Fingerprint Dive into the research topics of 'Automatic video annotation using multimodal Dirichlet process mixture model'. Together they form a unique fingerprint.

Cite this