Building models of animals from video

Deva Ramanan, David Alexander Forsyth, Kobus Barnard

Research output: Contribution to journalArticle

Abstract

This paper argues that tracking, object detection, and model building are all similar activities. We describe a fully automatic system that builds 2D articulated models known as pictorial structures from videos of animals. The learned model can be used to detect the animal in the original video - in this sense, the system can be viewed as a generalized tracker (one that is capable of modeling objects while tracking them). The learned model can be matched to a visual library; here, the system can be viewed as a video recognition algorithm. The learned model can also be used to detect the animal in novel images - in this case, the system can be seen as a method for learning models for object recognition. We find that we can significantly improve the pictorial structures by augmenting them with a discriminative texture model learned from a texture library. We develop a novel texture descriptor that outperforms the state-of-the-art for animal textures. We demonstrate the entire system on real video sequences of three different animals. We show that we can automatically track and identify the given animal. We use the learned models to recognize animals from two data sets; images taken by professional photographers from the Corel collection, and assorted images from the Web returned by Google. We demonstrate quite good performance on both data sets. Comparing our results with simple baselines, we show that, for the Google set, we can detect, localize, and recover part articulations from a collection demonstrably hard for object recognition.

Original languageEnglish (US)
Pages (from-to)1319-1333
Number of pages15
JournalIEEE transactions on pattern analysis and machine intelligence
Volume28
Issue number8
DOIs
StatePublished - Dec 1 2006

Fingerprint

Animals
Texture
Textures
Object Recognition
Object recognition
Model
Object Model
Object Tracking
Object Detection
Recognition Algorithm
Demonstrate
Descriptors
Baseline
Entire
Modeling

Keywords

  • Object recognition
  • Shape
  • Texture
  • Tracking
  • Video analysis

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Cite this

Building models of animals from video. / Ramanan, Deva; Forsyth, David Alexander; Barnard, Kobus.

In: IEEE transactions on pattern analysis and machine intelligence, Vol. 28, No. 8, 01.12.2006, p. 1319-1333.

Research output: Contribution to journalArticle

@article{6e04928780574664b9bd7a54e7612fe1,
title = "Building models of animals from video",
abstract = "This paper argues that tracking, object detection, and model building are all similar activities. We describe a fully automatic system that builds 2D articulated models known as pictorial structures from videos of animals. The learned model can be used to detect the animal in the original video - in this sense, the system can be viewed as a generalized tracker (one that is capable of modeling objects while tracking them). The learned model can be matched to a visual library; here, the system can be viewed as a video recognition algorithm. The learned model can also be used to detect the animal in novel images - in this case, the system can be seen as a method for learning models for object recognition. We find that we can significantly improve the pictorial structures by augmenting them with a discriminative texture model learned from a texture library. We develop a novel texture descriptor that outperforms the state-of-the-art for animal textures. We demonstrate the entire system on real video sequences of three different animals. We show that we can automatically track and identify the given animal. We use the learned models to recognize animals from two data sets; images taken by professional photographers from the Corel collection, and assorted images from the Web returned by Google. We demonstrate quite good performance on both data sets. Comparing our results with simple baselines, we show that, for the Google set, we can detect, localize, and recover part articulations from a collection demonstrably hard for object recognition.",
keywords = "Object recognition, Shape, Texture, Tracking, Video analysis",
author = "Deva Ramanan and Forsyth, {David Alexander} and Kobus Barnard",
year = "2006",
month = "12",
day = "1",
doi = "10.1109/TPAMI.2006.155",
language = "English (US)",
volume = "28",
pages = "1319--1333",
journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
issn = "0162-8828",
publisher = "IEEE Computer Society",
number = "8",

}

TY - JOUR

T1 - Building models of animals from video

AU - Ramanan, Deva

AU - Forsyth, David Alexander

AU - Barnard, Kobus

PY - 2006/12/1

Y1 - 2006/12/1

N2 - This paper argues that tracking, object detection, and model building are all similar activities. We describe a fully automatic system that builds 2D articulated models known as pictorial structures from videos of animals. The learned model can be used to detect the animal in the original video - in this sense, the system can be viewed as a generalized tracker (one that is capable of modeling objects while tracking them). The learned model can be matched to a visual library; here, the system can be viewed as a video recognition algorithm. The learned model can also be used to detect the animal in novel images - in this case, the system can be seen as a method for learning models for object recognition. We find that we can significantly improve the pictorial structures by augmenting them with a discriminative texture model learned from a texture library. We develop a novel texture descriptor that outperforms the state-of-the-art for animal textures. We demonstrate the entire system on real video sequences of three different animals. We show that we can automatically track and identify the given animal. We use the learned models to recognize animals from two data sets; images taken by professional photographers from the Corel collection, and assorted images from the Web returned by Google. We demonstrate quite good performance on both data sets. Comparing our results with simple baselines, we show that, for the Google set, we can detect, localize, and recover part articulations from a collection demonstrably hard for object recognition.

AB - This paper argues that tracking, object detection, and model building are all similar activities. We describe a fully automatic system that builds 2D articulated models known as pictorial structures from videos of animals. The learned model can be used to detect the animal in the original video - in this sense, the system can be viewed as a generalized tracker (one that is capable of modeling objects while tracking them). The learned model can be matched to a visual library; here, the system can be viewed as a video recognition algorithm. The learned model can also be used to detect the animal in novel images - in this case, the system can be seen as a method for learning models for object recognition. We find that we can significantly improve the pictorial structures by augmenting them with a discriminative texture model learned from a texture library. We develop a novel texture descriptor that outperforms the state-of-the-art for animal textures. We demonstrate the entire system on real video sequences of three different animals. We show that we can automatically track and identify the given animal. We use the learned models to recognize animals from two data sets; images taken by professional photographers from the Corel collection, and assorted images from the Web returned by Google. We demonstrate quite good performance on both data sets. Comparing our results with simple baselines, we show that, for the Google set, we can detect, localize, and recover part articulations from a collection demonstrably hard for object recognition.

KW - Object recognition

KW - Shape

KW - Texture

KW - Tracking

KW - Video analysis

UR - http://www.scopus.com/inward/record.url?scp=33748128037&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33748128037&partnerID=8YFLogxK

U2 - 10.1109/TPAMI.2006.155

DO - 10.1109/TPAMI.2006.155

M3 - Article

C2 - 16886866

AN - SCOPUS:33748128037

VL - 28

SP - 1319

EP - 1333

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

SN - 0162-8828

IS - 8

ER -