Representations and techniques for 3D object recognition and scene interpretation

Derek Hoiem, Silvio Savarese

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

One of the grand challenges of artificial intelligence is to enable computers to interpret 3D scenes and objects from imagery. This book organizes and introduces major concepts in 3D scene and object representation and inference from still images, with a focus on recent efforts to fuse models of geometry and perspective with statistical machine learning. The book is organized into three sections: (1) Interpretation of Physical Space; (2) Recognition of 3D Objects; and (3) Integrated 3D Scene Interpretation. The first discusses representations of spatial layout and techniques to interpret physical scenes from images. The second section introduces representations for 3D object categories that account for the intrinsically 3D nature of objects and provide robustness to change in viewpoints. The third section discusses strategies to unite inference of scene geometry and object pose and identity into a coherent scene interpretation. Each section broadly surveys important ideas from cognitive science and artificial intelligence research, organizes and discusses key concepts and techniques from recent work in computer vision, and describes a few sample approaches in detail. Newcomers to computer vision will benefit from introductions to basic concepts, such as single-view geometry and image classification, while experts and novices alike may find inspiration from the book's organization and discussion of the most recent ideas in 3D scene understanding and 3D object recognition. Specific topics include: mathematics of perspective geometry; visual elements of the physical scene, structural 3D scene representations; techniques and features for image and region categorization; historical perspective, computational models, and datasets and machine learning techniques for 3D object recognition; inferences of geometrical attributes of objects, such as size and pose; and probabilistic and feature-passing approaches for contextual reasoning about 3D objects and scenes.

Original languageEnglish (US)
Title of host publicationRepresentations and Techniques for 3D Object Recognition and Scene Interpretation
Pages1-169
Number of pages169
DOIs
StatePublished - Aug 18 2011

Publication series

NameSynthesis Lectures on Artificial Intelligence and Machine Learning
Volume15
ISSN (Print)1939-4608
ISSN (Electronic)1939-4616

Fingerprint

Object recognition
Geometry
Computer vision
Artificial intelligence
Learning systems
Image classification
Electric fuses

Keywords

  • 3D object models
  • 3D scene models
  • computer vision
  • context
  • image categorization
  • object recognition
  • scene understanding
  • single-view geometry

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Hoiem, D., & Savarese, S. (2011). Representations and techniques for 3D object recognition and scene interpretation. In Representations and Techniques for 3D Object Recognition and Scene Interpretation (pp. 1-169). (Synthesis Lectures on Artificial Intelligence and Machine Learning; Vol. 15). https://doi.org/10.2200/S00370ED1V01Y201107AIM015

Representations and techniques for 3D object recognition and scene interpretation. / Hoiem, Derek; Savarese, Silvio.

Representations and Techniques for 3D Object Recognition and Scene Interpretation. 2011. p. 1-169 (Synthesis Lectures on Artificial Intelligence and Machine Learning; Vol. 15).

Research output: Chapter in Book/Report/Conference proceedingChapter

Hoiem, D & Savarese, S 2011, Representations and techniques for 3D object recognition and scene interpretation. in Representations and Techniques for 3D Object Recognition and Scene Interpretation. Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 15, pp. 1-169. https://doi.org/10.2200/S00370ED1V01Y201107AIM015
Hoiem D, Savarese S. Representations and techniques for 3D object recognition and scene interpretation. In Representations and Techniques for 3D Object Recognition and Scene Interpretation. 2011. p. 1-169. (Synthesis Lectures on Artificial Intelligence and Machine Learning). https://doi.org/10.2200/S00370ED1V01Y201107AIM015
Hoiem, Derek ; Savarese, Silvio. / Representations and techniques for 3D object recognition and scene interpretation. Representations and Techniques for 3D Object Recognition and Scene Interpretation. 2011. pp. 1-169 (Synthesis Lectures on Artificial Intelligence and Machine Learning).
@inbook{e687e0cbe3f64675a08305bf747f7c98,
title = "Representations and techniques for 3D object recognition and scene interpretation",
abstract = "One of the grand challenges of artificial intelligence is to enable computers to interpret 3D scenes and objects from imagery. This book organizes and introduces major concepts in 3D scene and object representation and inference from still images, with a focus on recent efforts to fuse models of geometry and perspective with statistical machine learning. The book is organized into three sections: (1) Interpretation of Physical Space; (2) Recognition of 3D Objects; and (3) Integrated 3D Scene Interpretation. The first discusses representations of spatial layout and techniques to interpret physical scenes from images. The second section introduces representations for 3D object categories that account for the intrinsically 3D nature of objects and provide robustness to change in viewpoints. The third section discusses strategies to unite inference of scene geometry and object pose and identity into a coherent scene interpretation. Each section broadly surveys important ideas from cognitive science and artificial intelligence research, organizes and discusses key concepts and techniques from recent work in computer vision, and describes a few sample approaches in detail. Newcomers to computer vision will benefit from introductions to basic concepts, such as single-view geometry and image classification, while experts and novices alike may find inspiration from the book's organization and discussion of the most recent ideas in 3D scene understanding and 3D object recognition. Specific topics include: mathematics of perspective geometry; visual elements of the physical scene, structural 3D scene representations; techniques and features for image and region categorization; historical perspective, computational models, and datasets and machine learning techniques for 3D object recognition; inferences of geometrical attributes of objects, such as size and pose; and probabilistic and feature-passing approaches for contextual reasoning about 3D objects and scenes.",
keywords = "3D object models, 3D scene models, computer vision, context, image categorization, object recognition, scene understanding, single-view geometry",
author = "Derek Hoiem and Silvio Savarese",
year = "2011",
month = "8",
day = "18",
doi = "10.2200/S00370ED1V01Y201107AIM015",
language = "English (US)",
isbn = "9781608457281",
series = "Synthesis Lectures on Artificial Intelligence and Machine Learning",
pages = "1--169",
booktitle = "Representations and Techniques for 3D Object Recognition and Scene Interpretation",

}

TY - CHAP

T1 - Representations and techniques for 3D object recognition and scene interpretation

AU - Hoiem, Derek

AU - Savarese, Silvio

PY - 2011/8/18

Y1 - 2011/8/18

N2 - One of the grand challenges of artificial intelligence is to enable computers to interpret 3D scenes and objects from imagery. This book organizes and introduces major concepts in 3D scene and object representation and inference from still images, with a focus on recent efforts to fuse models of geometry and perspective with statistical machine learning. The book is organized into three sections: (1) Interpretation of Physical Space; (2) Recognition of 3D Objects; and (3) Integrated 3D Scene Interpretation. The first discusses representations of spatial layout and techniques to interpret physical scenes from images. The second section introduces representations for 3D object categories that account for the intrinsically 3D nature of objects and provide robustness to change in viewpoints. The third section discusses strategies to unite inference of scene geometry and object pose and identity into a coherent scene interpretation. Each section broadly surveys important ideas from cognitive science and artificial intelligence research, organizes and discusses key concepts and techniques from recent work in computer vision, and describes a few sample approaches in detail. Newcomers to computer vision will benefit from introductions to basic concepts, such as single-view geometry and image classification, while experts and novices alike may find inspiration from the book's organization and discussion of the most recent ideas in 3D scene understanding and 3D object recognition. Specific topics include: mathematics of perspective geometry; visual elements of the physical scene, structural 3D scene representations; techniques and features for image and region categorization; historical perspective, computational models, and datasets and machine learning techniques for 3D object recognition; inferences of geometrical attributes of objects, such as size and pose; and probabilistic and feature-passing approaches for contextual reasoning about 3D objects and scenes.

AB - One of the grand challenges of artificial intelligence is to enable computers to interpret 3D scenes and objects from imagery. This book organizes and introduces major concepts in 3D scene and object representation and inference from still images, with a focus on recent efforts to fuse models of geometry and perspective with statistical machine learning. The book is organized into three sections: (1) Interpretation of Physical Space; (2) Recognition of 3D Objects; and (3) Integrated 3D Scene Interpretation. The first discusses representations of spatial layout and techniques to interpret physical scenes from images. The second section introduces representations for 3D object categories that account for the intrinsically 3D nature of objects and provide robustness to change in viewpoints. The third section discusses strategies to unite inference of scene geometry and object pose and identity into a coherent scene interpretation. Each section broadly surveys important ideas from cognitive science and artificial intelligence research, organizes and discusses key concepts and techniques from recent work in computer vision, and describes a few sample approaches in detail. Newcomers to computer vision will benefit from introductions to basic concepts, such as single-view geometry and image classification, while experts and novices alike may find inspiration from the book's organization and discussion of the most recent ideas in 3D scene understanding and 3D object recognition. Specific topics include: mathematics of perspective geometry; visual elements of the physical scene, structural 3D scene representations; techniques and features for image and region categorization; historical perspective, computational models, and datasets and machine learning techniques for 3D object recognition; inferences of geometrical attributes of objects, such as size and pose; and probabilistic and feature-passing approaches for contextual reasoning about 3D objects and scenes.

KW - 3D object models

KW - 3D scene models

KW - computer vision

KW - context

KW - image categorization

KW - object recognition

KW - scene understanding

KW - single-view geometry

UR - http://www.scopus.com/inward/record.url?scp=80051955582&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80051955582&partnerID=8YFLogxK

U2 - 10.2200/S00370ED1V01Y201107AIM015

DO - 10.2200/S00370ED1V01Y201107AIM015

M3 - Chapter

AN - SCOPUS:80051955582

SN - 9781608457281

T3 - Synthesis Lectures on Artificial Intelligence and Machine Learning

SP - 1

EP - 169

BT - Representations and Techniques for 3D Object Recognition and Scene Interpretation

ER -