Building text features for object image classification

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We introduce a text-based image feature and demon- strate that it consistently improves performance on hard object classification problems. The feature is built using an auxiliary dataset of images annotated with tags, down- loaded from the internet. We do not inspect or correct the tags and expect that they are noisy. We obtain the text fea- ture of an unannotated image from the tags of its k-nearest neighbors in this auxiliary collection. A visual classifier presented with an object viewed un- der novel circumstances (say, a new viewing direction) must rely on its visual examples. Our text feature may not change, because the auxiliary dataset likely contains a similar pic- ture. While the tags associated with images are noisy, they are more stable when appearance changes. We test the performance of this feature using PAS- CAL VOC 2006 and 2007 datasets. Our feature performs well, consistently improves the performance of visual ob- ject classifiers, and is particularly effective when the train- ing dataset is small.

Original languageEnglish (US)
Title of host publication2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009
PublisherIEEE Computer Society
Pages1367-1374
Number of pages8
ISBN (Print)9781424439935
DOIs
StatePublished - Jan 1 2009
Event2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Miami, FL, United States
Duration: Jun 20 2009Jun 25 2009

Publication series

Name2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009
Volume2009 IEEE Computer Society Conference on Computer Vision and ...

Other

Other2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
CountryUnited States
CityMiami, FL
Period6/20/096/25/09

Fingerprint

Image classification
Classifiers
Volatile organic compounds
Internet

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Biomedical Engineering

Cite this

Wang, G., Hoiem, D. W., & Forsyth, D. A. (2009). Building text features for object image classification. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009 (pp. 1367-1374). [5206816] (2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009; Vol. 2009 IEEE Computer Society Conference on Computer Vision and ...). IEEE Computer Society. https://doi.org/10.1109/CVPRW.2009.5206816

Building text features for object image classification. / Wang, Gang; Hoiem, Derek W; Forsyth, David Alexander.

2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009. IEEE Computer Society, 2009. p. 1367-1374 5206816 (2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009; Vol. 2009 IEEE Computer Society Conference on Computer Vision and ...).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, G, Hoiem, DW & Forsyth, DA 2009, Building text features for object image classification. in 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009., 5206816, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009, vol. 2009 IEEE Computer Society Conference on Computer Vision and ..., IEEE Computer Society, pp. 1367-1374, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Miami, FL, United States, 6/20/09. https://doi.org/10.1109/CVPRW.2009.5206816
Wang G, Hoiem DW, Forsyth DA. Building text features for object image classification. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009. IEEE Computer Society. 2009. p. 1367-1374. 5206816. (2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009). https://doi.org/10.1109/CVPRW.2009.5206816
Wang, Gang ; Hoiem, Derek W ; Forsyth, David Alexander. / Building text features for object image classification. 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009. IEEE Computer Society, 2009. pp. 1367-1374 (2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009).
@inproceedings{148d986a6dc040738e924f5a3a7e0955,
title = "Building text features for object image classification",
abstract = "We introduce a text-based image feature and demon- strate that it consistently improves performance on hard object classification problems. The feature is built using an auxiliary dataset of images annotated with tags, down- loaded from the internet. We do not inspect or correct the tags and expect that they are noisy. We obtain the text fea- ture of an unannotated image from the tags of its k-nearest neighbors in this auxiliary collection. A visual classifier presented with an object viewed un- der novel circumstances (say, a new viewing direction) must rely on its visual examples. Our text feature may not change, because the auxiliary dataset likely contains a similar pic- ture. While the tags associated with images are noisy, they are more stable when appearance changes. We test the performance of this feature using PAS- CAL VOC 2006 and 2007 datasets. Our feature performs well, consistently improves the performance of visual ob- ject classifiers, and is particularly effective when the train- ing dataset is small.",
author = "Gang Wang and Hoiem, {Derek W} and Forsyth, {David Alexander}",
year = "2009",
month = "1",
day = "1",
doi = "10.1109/CVPRW.2009.5206816",
language = "English (US)",
isbn = "9781424439935",
series = "2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009",
publisher = "IEEE Computer Society",
pages = "1367--1374",
booktitle = "2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009",

}

TY - GEN

T1 - Building text features for object image classification

AU - Wang, Gang

AU - Hoiem, Derek W

AU - Forsyth, David Alexander

PY - 2009/1/1

Y1 - 2009/1/1

N2 - We introduce a text-based image feature and demon- strate that it consistently improves performance on hard object classification problems. The feature is built using an auxiliary dataset of images annotated with tags, down- loaded from the internet. We do not inspect or correct the tags and expect that they are noisy. We obtain the text fea- ture of an unannotated image from the tags of its k-nearest neighbors in this auxiliary collection. A visual classifier presented with an object viewed un- der novel circumstances (say, a new viewing direction) must rely on its visual examples. Our text feature may not change, because the auxiliary dataset likely contains a similar pic- ture. While the tags associated with images are noisy, they are more stable when appearance changes. We test the performance of this feature using PAS- CAL VOC 2006 and 2007 datasets. Our feature performs well, consistently improves the performance of visual ob- ject classifiers, and is particularly effective when the train- ing dataset is small.

AB - We introduce a text-based image feature and demon- strate that it consistently improves performance on hard object classification problems. The feature is built using an auxiliary dataset of images annotated with tags, down- loaded from the internet. We do not inspect or correct the tags and expect that they are noisy. We obtain the text fea- ture of an unannotated image from the tags of its k-nearest neighbors in this auxiliary collection. A visual classifier presented with an object viewed un- der novel circumstances (say, a new viewing direction) must rely on its visual examples. Our text feature may not change, because the auxiliary dataset likely contains a similar pic- ture. While the tags associated with images are noisy, they are more stable when appearance changes. We test the performance of this feature using PAS- CAL VOC 2006 and 2007 datasets. Our feature performs well, consistently improves the performance of visual ob- ject classifiers, and is particularly effective when the train- ing dataset is small.

UR - http://www.scopus.com/inward/record.url?scp=70450207253&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70450207253&partnerID=8YFLogxK

U2 - 10.1109/CVPRW.2009.5206816

DO - 10.1109/CVPRW.2009.5206816

M3 - Conference contribution

AN - SCOPUS:70450207253

SN - 9781424439935

T3 - 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009

SP - 1367

EP - 1374

BT - 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009

PB - IEEE Computer Society

ER -