Weakly supervised fine-grained categorization with part-based image representation

Yu Zhang, Xiu Shen Wei, Jianxin Wu, Jianfei Cai, Jiangbo Lu, Viet Anh Nguyen, Minh N. Do

Research output: Contribution to journalArticlepeer-review


In this paper, we propose a fine-grained image categorization system with easy deployment. We do not use any object/part annotation (weakly supervised) in the training or in the testing stage, but only class labels for training images. Fine-grained image categorization aims to classify objects with only subtle distinctions (e.g., two breeds of dogs that look alike). Most existing works heavily rely on object/part detectors to build the correspondence between object parts, which require accurate object or object part annotations at least for training images. The need for expensive object annotations prevents the wide usage of these methods. Instead, we propose to generate multi-scale part proposals from object proposals, select useful part proposals, and use them to compute a global image representation for categorization. This is specially designed for the weakly supervised fine-grained categorization task, because useful parts have been shown to play a critical role in existing annotation-dependent works, but accurate part detectors are hard to acquire. With the proposed image representation, we can further detect and visualize the key (most discriminative) parts in objects of different classes. In the experiments, the proposed weakly supervised method achieves comparable or better accuracy than the state-of-the-art weakly supervised methods and most existing annotation-dependent methods on three challenging datasets. Its success suggests that it is not always necessary to learn expensive object/part detectors in fine-grained image categorization.

Original languageEnglish (US)
Article number7410088
Pages (from-to)1713-1725
Number of pages13
JournalIEEE Transactions on Image Processing
Issue number4
StatePublished - Apr 2016


  • Fine grained categorization
  • part selection
  • weakly supervised

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Weakly supervised fine-grained categorization with part-based image representation'. Together they form a unique fingerprint.

Cite this