Building a dictionary of image fragments

Zicheng Liao, Ali Farhadi, Yang Wang, Ian Endres, David Alexander Forsyth

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We show how to build large dictionaries of meaningful image fragments. These fragments could represent objects, objects in a local context, or parts of scenes. Our fragments operate as region-based exemplars, and we show how they can be used for image classification, to localize objects, and to compose new images. While each of these activities has been demonstrated before, each has required manually extracted fragments. Because our method for fragment extraction is automatic it can operate at a large scale. Our method uses recent advances in generic object detection techniques, together with discriminative tests to obtain good, clean fragment sets with extensive diversity. Our fragments are organized by the tags of the source images to build a semantically organized fragment table. A good set of fragment exemplars describes only the object, rather than objectcontext. Context could help identify an object; but it could also contribute noise, because other objects might appear in the same context. We show a slight improvement in classification performance by two standard exemplar matching methods using our fragment dictionary over such methods using image exemplars. This suggests that knowing the support of an exemplar is valuable. Furthermore, we demonstrate our automatically built fragment dictionary is capable of good localization. Finally, our fragment dictionary supports a keyword based fragment search system, which allows artists to get the fragments they need to make image collages.

Original languageEnglish (US)
Title of host publication2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012
Pages3442-3449
Number of pages8
DOIs
StatePublished - Oct 1 2012
Event2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012 - Providence, RI, United States
Duration: Jun 16 2012Jun 21 2012

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN (Print)1063-6919

Other

Other2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012
CountryUnited States
CityProvidence, RI
Period6/16/126/21/12

Fingerprint

Glossaries
Image classification

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Liao, Z., Farhadi, A., Wang, Y., Endres, I., & Forsyth, D. A. (2012). Building a dictionary of image fragments. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012 (pp. 3442-3449). [6248085] (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition). https://doi.org/10.1109/CVPR.2012.6248085

Building a dictionary of image fragments. / Liao, Zicheng; Farhadi, Ali; Wang, Yang; Endres, Ian; Forsyth, David Alexander.

2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012. 2012. p. 3442-3449 6248085 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Liao, Z, Farhadi, A, Wang, Y, Endres, I & Forsyth, DA 2012, Building a dictionary of image fragments. in 2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012., 6248085, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3442-3449, 2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012, Providence, RI, United States, 6/16/12. https://doi.org/10.1109/CVPR.2012.6248085
Liao Z, Farhadi A, Wang Y, Endres I, Forsyth DA. Building a dictionary of image fragments. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012. 2012. p. 3442-3449. 6248085. (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition). https://doi.org/10.1109/CVPR.2012.6248085
Liao, Zicheng ; Farhadi, Ali ; Wang, Yang ; Endres, Ian ; Forsyth, David Alexander. / Building a dictionary of image fragments. 2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012. 2012. pp. 3442-3449 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition).
@inproceedings{d1d00fd6f475452080e3660791203d43,
title = "Building a dictionary of image fragments",
abstract = "We show how to build large dictionaries of meaningful image fragments. These fragments could represent objects, objects in a local context, or parts of scenes. Our fragments operate as region-based exemplars, and we show how they can be used for image classification, to localize objects, and to compose new images. While each of these activities has been demonstrated before, each has required manually extracted fragments. Because our method for fragment extraction is automatic it can operate at a large scale. Our method uses recent advances in generic object detection techniques, together with discriminative tests to obtain good, clean fragment sets with extensive diversity. Our fragments are organized by the tags of the source images to build a semantically organized fragment table. A good set of fragment exemplars describes only the object, rather than objectcontext. Context could help identify an object; but it could also contribute noise, because other objects might appear in the same context. We show a slight improvement in classification performance by two standard exemplar matching methods using our fragment dictionary over such methods using image exemplars. This suggests that knowing the support of an exemplar is valuable. Furthermore, we demonstrate our automatically built fragment dictionary is capable of good localization. Finally, our fragment dictionary supports a keyword based fragment search system, which allows artists to get the fragments they need to make image collages.",
author = "Zicheng Liao and Ali Farhadi and Yang Wang and Ian Endres and Forsyth, {David Alexander}",
year = "2012",
month = "10",
day = "1",
doi = "10.1109/CVPR.2012.6248085",
language = "English (US)",
isbn = "9781467312264",
series = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",
pages = "3442--3449",
booktitle = "2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012",

}

TY - GEN

T1 - Building a dictionary of image fragments

AU - Liao, Zicheng

AU - Farhadi, Ali

AU - Wang, Yang

AU - Endres, Ian

AU - Forsyth, David Alexander

PY - 2012/10/1

Y1 - 2012/10/1

N2 - We show how to build large dictionaries of meaningful image fragments. These fragments could represent objects, objects in a local context, or parts of scenes. Our fragments operate as region-based exemplars, and we show how they can be used for image classification, to localize objects, and to compose new images. While each of these activities has been demonstrated before, each has required manually extracted fragments. Because our method for fragment extraction is automatic it can operate at a large scale. Our method uses recent advances in generic object detection techniques, together with discriminative tests to obtain good, clean fragment sets with extensive diversity. Our fragments are organized by the tags of the source images to build a semantically organized fragment table. A good set of fragment exemplars describes only the object, rather than objectcontext. Context could help identify an object; but it could also contribute noise, because other objects might appear in the same context. We show a slight improvement in classification performance by two standard exemplar matching methods using our fragment dictionary over such methods using image exemplars. This suggests that knowing the support of an exemplar is valuable. Furthermore, we demonstrate our automatically built fragment dictionary is capable of good localization. Finally, our fragment dictionary supports a keyword based fragment search system, which allows artists to get the fragments they need to make image collages.

AB - We show how to build large dictionaries of meaningful image fragments. These fragments could represent objects, objects in a local context, or parts of scenes. Our fragments operate as region-based exemplars, and we show how they can be used for image classification, to localize objects, and to compose new images. While each of these activities has been demonstrated before, each has required manually extracted fragments. Because our method for fragment extraction is automatic it can operate at a large scale. Our method uses recent advances in generic object detection techniques, together with discriminative tests to obtain good, clean fragment sets with extensive diversity. Our fragments are organized by the tags of the source images to build a semantically organized fragment table. A good set of fragment exemplars describes only the object, rather than objectcontext. Context could help identify an object; but it could also contribute noise, because other objects might appear in the same context. We show a slight improvement in classification performance by two standard exemplar matching methods using our fragment dictionary over such methods using image exemplars. This suggests that knowing the support of an exemplar is valuable. Furthermore, we demonstrate our automatically built fragment dictionary is capable of good localization. Finally, our fragment dictionary supports a keyword based fragment search system, which allows artists to get the fragments they need to make image collages.

UR - http://www.scopus.com/inward/record.url?scp=84866706755&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84866706755&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2012.6248085

DO - 10.1109/CVPR.2012.6248085

M3 - Conference contribution

AN - SCOPUS:84866706755

SN - 9781467312264

T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SP - 3442

EP - 3449

BT - 2012 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2012

ER -