A visual annotation framework using common-sensical and linguistic relationships for semantic media retrieval

Bageshree Shevade, Hari Sundaram

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we present a novel image annotation approach with an emphasis on - (a) common sense based semantic propagation, (b) visual annotation interfaces and (c) novel evaluation schemes. The annotation system is interactive, intuitive and real-time. We attempt to propagate semantics of the annotations, by using WordNet and ConceptNet, and low-level features extracted from the images. We introduce novel semantic dissimilarity measures, and propagation frameworks. We develop a novel visual annotation interface that allows a user to group images by creating visual concepts using direct manipulation metaphors without manual annotation. We also develop a new evaluation technique for annotation that is based on relationship between concepts based on commonsensical relationships. Our Experimental results on three different datasets, indicate that the annotation system performs very well. The semantic propagation results are good - we converge close to the semantics of the image by annotating a small number (-16.8%) of database images.

Original languageEnglish (US)
Title of host publicationAdaptive Multimedia Retrieval
Subtitle of host publicationUser, Context, and Feedback - Third International Workshop, AMR 2005, Revised Selected Papers
Pages251-265
Number of pages15
DOIs
StatePublished - Jul 7 2006
Externally publishedYes
Event3rd International Workshop on Adaptive Multimedia Retrieval: User, Context, and Feedback, AMR 2005 - Glasgow, United Kingdom
Duration: Jul 28 2005Jul 29 2005

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3877 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other3rd International Workshop on Adaptive Multimedia Retrieval: User, Context, and Feedback, AMR 2005
Country/TerritoryUnited Kingdom
CityGlasgow
Period7/28/057/29/05

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'A visual annotation framework using common-sensical and linguistic relationships for semantic media retrieval'. Together they form a unique fingerprint.

Cite this