Extracting semantics from audiovisual content: The final frontier in multimedia retrieval

Milind R. Naphade, Thomas S. Huang

Research output: Contribution to journalArticlepeer-review


Multimedia understanding is a fast emerging interdisciplinary research area. There is tremendous potential for effective use of multimedia content through intelligent analysis. Diverse application areas are increasingly relying on multimedia understanding systems. Advances in multimedia understanding are related directly to advances in signal processing, computer vision, pattern recognition, multimedia databases, and smart sensors. We review the state-of-the-art techniques in multimedia retrieval. In particular we discuss how multimedia retrieval can be viewed as a pattern recognition problem. We discuss, how reliance on powerful pattern recognition and machine learning techniques is increasing in the field of multimedia retrieval. We review state-of-the-art multimedia understanding systems with particular emphasis on a system for semantic video indexing centered around multijects and multinets. We discuss how semantic retrieval is centered around concepts and context and also discuss various mechanisms for modeling concepts and context.

Original languageEnglish (US)
Pages (from-to)793-810
Number of pages18
JournalIEEE Transactions on Neural Networks
Issue number4
StatePublished - Jul 2002
Externally publishedYes


  • Bayesian networks
  • Decision theory
  • Factor graphs
  • Machine learning
  • Multijects
  • Multimedia understanding
  • Multinets
  • Semantic video indexing
  • Statistical pattern recognition
  • Sum-product algorithm

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence


Dive into the research topics of 'Extracting semantics from audiovisual content: The final frontier in multimedia retrieval'. Together they form a unique fingerprint.

Cite this