Unsupervised visual representation learning by graph-based consistent constraints

Dong Li, Wei Chih Hung, Jia Bin Huang, Shengjin Wang, Narendra Ahuja, Ming Hsuan Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Learning rich visual representations often require training on datasets of millions of manually annotated examples. This substantially limits the scalability of learning effective representations as labeled data is expensive or scarce. In this paper, we address the problem of unsupervised visual representation learning from a large, unlabeled collection of images. By representing each image as a node and each nearest-neighbor matching pair as an edge, our key idea is to leverage graph-based analysis to discover positive and negative image pairs (i.e., pairs belonging to the same and different visual categories). Specifically, we propose to use a cycle consistency criterion for mining positive pairs and geodesic distance in the graph for hard negative mining. We show that the mined positive and negative image pairs can provide accurate supervisory signals for learning effective representations using Convolutional Neural Networks (CNNs). We demonstrate the effectiveness of the proposed unsupervised constraint mining method in two settings: (1) unsupervised feature learning and (2) semi-supervised learning. For unsupervised feature learning, we obtain competitive performance with several state-of-the-art approaches on the PASCAL VOC 2007 dataset. For semisupervised learning, we show boosted performance by incorporating the mined constraints on three image classification datasets.

Original languageEnglish (US)
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
EditorsBastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
Number of pages17
ISBN (Print)9783319464923
StatePublished - 2016
Event14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands
Duration: Oct 11 2016Oct 14 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9908 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other14th European Conference on Computer Vision, ECCV 2016


  • Convolutional neural networks
  • Image classification
  • Semi-supervised learning
  • Unsupervised feature learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Unsupervised visual representation learning by graph-based consistent constraints'. Together they form a unique fingerprint.

Cite this