Contextual wisdom: Social relations and correlations for multimedia event annotation

Amit Zunjarwad, Hari Sundaram, Lexing Xie

Research output: Chapter in Book/Report/Conference proceedingConference contribution


This work deals with the problem of event annotation in social networks. The problem is made difficult due to variability of semantics and due to scarcity of labeled data. Events refer to real-world phenomena that occur at a specific time and place, and media and text tags are treated as facets of the event metadata. We are proposing a novel mechanism for event annotation by leveraging related sources (other annotators) in a social network. Our approach exploits event concept similarity, concept co-occurrence and annotator trust. We compute concept similarity measures across all facets. These measures are then used to compute event-event and user-user activity correlation. We compute inter-facet concept co-occurrence statistics from the annotations by each user. The annotator trust is determined by first requesting the trusted annotators (seeds) from each user and then propagating the trust amongst the social network using the biased PageRank algorithm. For a specific media instance to be annotated, we start the process from an initial query vector and the optimal recommendations are determined by using a coupling strategy between the global similarity matrix, and the trust weighted global co-occurrence matrix. The coupling links the common shared knowledge (similarity between concepts) that exists within the social network with trusted and personalized observations (concept co-occurrences). Our initial experiments on annotated everyday events are promising and show substantial gains against traditional SVM based techniques.

Original languageEnglish (US)
Title of host publicationProceedings of the Fifteenth ACM International Conference on Multimedia, MM'07
Number of pages10
StatePublished - 2007
Externally publishedYes
Event15th ACM International Conference on Multimedia, MM'07 - Augsburg, Bavaria, Germany
Duration: Sep 24 2007Sep 29 2007

Publication series

NameProceedings of the ACM International Multimedia Conference and Exhibition


Other15th ACM International Conference on Multimedia, MM'07
CityAugsburg, Bavaria


  • Content management
  • Context
  • Event annotation
  • Images
  • Multimedia
  • Social networks

ASJC Scopus subject areas

  • Computer Science(all)


Dive into the research topics of 'Contextual wisdom: Social relations and correlations for multimedia event annotation'. Together they form a unique fingerprint.

Cite this