Learning rich features from RGB-D images for object detection and segmentation

Saurabh Gupta, Ross Girshick, Pablo Arbeláez, Jitendra Malik

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper we study the problem of object detection for RGB-D images using semantically rich image and depth features. We propose a new geocentric embedding for depth images that encodes height above ground and angle with gravity for each pixel in addition to the horizontal disparity. We demonstrate that this geocentric embedding works better than using raw depth images for learning feature representations with convolutional neural networks. Our final object detection system achieves an average precision of 37.3%, which is a 56% relative improvement over existing methods. We then focus on the task of instance segmentation where we label pixels belonging to object instances found by our detector. For this task, we propose a decision forest approach that classifies pixels in the detection window as foreground or background using a family of unary and binary tests that query shape and geocentric pose features. Finally, we use the output from our object detectors in an existing superpixel classification framework for semantic scene segmentation and achieve a 24% relative improvement over current state-of-the-art for the object categories that we study. We believe advances such as those represented in this paper will facilitate the use of perception in fields like robotics.

Original languageEnglish (US)
Title of host publicationComputer Vision, ECCV 2014 - 13th European Conference, Proceedings
PublisherSpringer-Verlag Berlin Heidelberg
Pages345-360
Number of pages16
EditionPART 7
ISBN (Print)9783319105833
DOIs
StatePublished - Jan 1 2014
Externally publishedYes
Event13th European Conference on Computer Vision, ECCV 2014 - Zurich, Switzerland
Duration: Sep 6 2014Sep 12 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 7
Volume8695 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other13th European Conference on Computer Vision, ECCV 2014
CountrySwitzerland
CityZurich
Period9/6/149/12/14

Keywords

  • RGB-D perception
  • object detection
  • object segmentation

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Learning rich features from RGB-D images for object detection and segmentation'. Together they form a unique fingerprint.

Cite this