3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks

Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, Derek W Hoiem

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The success of various applications including robotics, digital content creation, and visualization demand a structured and abstract representation of the 3D world from limited sensor data. Inspired by the nature of human perception of 3D shapes as a collection of simple parts, we explore such an abstract shape representation based on primitives. Given a single depth image of an object, we present 3DPRNN, a generative recurrent neural network that synthesizes multiple plausible shapes composed of a set of primitives. Our generative model encodes symmetry characteristics of common man-made objects, preserves long-range structural coherence, and describes objects of varying complexity with a compact representation. We also propose a method based on Gaussian Fields to generate a large scale dataset of primitive-based shape representations to train our network. We evaluate our approach on a wide range of examples and show that it outperforms nearest-neighbor based shape retrieval methods and is on-par with voxelbased generative models while using a significantly reduced parameter space.

Original languageEnglish (US)
Title of host publicationProceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages900-909
Number of pages10
ISBN (Electronic)9781538610329
DOIs
StatePublished - Dec 22 2017
Event16th IEEE International Conference on Computer Vision, ICCV 2017 - Venice, Italy
Duration: Oct 22 2017Oct 29 2017

Publication series

NameProceedings of the IEEE International Conference on Computer Vision
Volume2017-October
ISSN (Print)1550-5499

Other

Other16th IEEE International Conference on Computer Vision, ICCV 2017
CountryItaly
CityVenice
Period10/22/1710/29/17

Fingerprint

Recurrent neural networks
Robotics
Visualization
Sensors

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Zou, C., Yumer, E., Yang, J., Ceylan, D., & Hoiem, D. W. (2017). 3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks. In Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017 (pp. 900-909). [8237365] (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2017-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2017.103

3D-PRNN : Generating Shape Primitives with Recurrent Neural Networks. / Zou, Chuhang; Yumer, Ersin; Yang, Jimei; Ceylan, Duygu; Hoiem, Derek W.

Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 900-909 8237365 (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2017-October).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zou, C, Yumer, E, Yang, J, Ceylan, D & Hoiem, DW 2017, 3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks. in Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017., 8237365, Proceedings of the IEEE International Conference on Computer Vision, vol. 2017-October, Institute of Electrical and Electronics Engineers Inc., pp. 900-909, 16th IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 10/22/17. https://doi.org/10.1109/ICCV.2017.103
Zou C, Yumer E, Yang J, Ceylan D, Hoiem DW. 3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks. In Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 900-909. 8237365. (Proceedings of the IEEE International Conference on Computer Vision). https://doi.org/10.1109/ICCV.2017.103
Zou, Chuhang ; Yumer, Ersin ; Yang, Jimei ; Ceylan, Duygu ; Hoiem, Derek W. / 3D-PRNN : Generating Shape Primitives with Recurrent Neural Networks. Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 900-909 (Proceedings of the IEEE International Conference on Computer Vision).
@inproceedings{c9b89ceb731740518ba698c4a612ec4a,
title = "3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks",
abstract = "The success of various applications including robotics, digital content creation, and visualization demand a structured and abstract representation of the 3D world from limited sensor data. Inspired by the nature of human perception of 3D shapes as a collection of simple parts, we explore such an abstract shape representation based on primitives. Given a single depth image of an object, we present 3DPRNN, a generative recurrent neural network that synthesizes multiple plausible shapes composed of a set of primitives. Our generative model encodes symmetry characteristics of common man-made objects, preserves long-range structural coherence, and describes objects of varying complexity with a compact representation. We also propose a method based on Gaussian Fields to generate a large scale dataset of primitive-based shape representations to train our network. We evaluate our approach on a wide range of examples and show that it outperforms nearest-neighbor based shape retrieval methods and is on-par with voxelbased generative models while using a significantly reduced parameter space.",
author = "Chuhang Zou and Ersin Yumer and Jimei Yang and Duygu Ceylan and Hoiem, {Derek W}",
year = "2017",
month = "12",
day = "22",
doi = "10.1109/ICCV.2017.103",
language = "English (US)",
series = "Proceedings of the IEEE International Conference on Computer Vision",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "900--909",
booktitle = "Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017",
address = "United States",

}

TY - GEN

T1 - 3D-PRNN

T2 - Generating Shape Primitives with Recurrent Neural Networks

AU - Zou, Chuhang

AU - Yumer, Ersin

AU - Yang, Jimei

AU - Ceylan, Duygu

AU - Hoiem, Derek W

PY - 2017/12/22

Y1 - 2017/12/22

N2 - The success of various applications including robotics, digital content creation, and visualization demand a structured and abstract representation of the 3D world from limited sensor data. Inspired by the nature of human perception of 3D shapes as a collection of simple parts, we explore such an abstract shape representation based on primitives. Given a single depth image of an object, we present 3DPRNN, a generative recurrent neural network that synthesizes multiple plausible shapes composed of a set of primitives. Our generative model encodes symmetry characteristics of common man-made objects, preserves long-range structural coherence, and describes objects of varying complexity with a compact representation. We also propose a method based on Gaussian Fields to generate a large scale dataset of primitive-based shape representations to train our network. We evaluate our approach on a wide range of examples and show that it outperforms nearest-neighbor based shape retrieval methods and is on-par with voxelbased generative models while using a significantly reduced parameter space.

AB - The success of various applications including robotics, digital content creation, and visualization demand a structured and abstract representation of the 3D world from limited sensor data. Inspired by the nature of human perception of 3D shapes as a collection of simple parts, we explore such an abstract shape representation based on primitives. Given a single depth image of an object, we present 3DPRNN, a generative recurrent neural network that synthesizes multiple plausible shapes composed of a set of primitives. Our generative model encodes symmetry characteristics of common man-made objects, preserves long-range structural coherence, and describes objects of varying complexity with a compact representation. We also propose a method based on Gaussian Fields to generate a large scale dataset of primitive-based shape representations to train our network. We evaluate our approach on a wide range of examples and show that it outperforms nearest-neighbor based shape retrieval methods and is on-par with voxelbased generative models while using a significantly reduced parameter space.

UR - http://www.scopus.com/inward/record.url?scp=85041917778&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041917778&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2017.103

DO - 10.1109/ICCV.2017.103

M3 - Conference contribution

AN - SCOPUS:85041917778

T3 - Proceedings of the IEEE International Conference on Computer Vision

SP - 900

EP - 909

BT - Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -