Manhattan Room Layout Reconstruction from a Single 360 Image: A Comparative Study of State-of-the-Art Methods

Chuhang Zou, Jheng Wei Su, Chi Han Peng, Alex Colburn, Qi Shan, Peter Wonka, Hung Kuo Chu, Derek Hoiem

Research output: Contribution to journalArticlepeer-review

Abstract

Recent approaches for predicting layouts from 360 panoramas produce excellent results. These approaches build on a common framework consisting of three steps: a pre-processing step based on edge-based alignment, prediction of layout elements, and a post-processing step by fitting a 3D layout to the layout elements. Until now, it has been difficult to compare the methods due to multiple different design decisions, such as the encoding network (e.g., SegNet or ResNet), type of elements predicted (e.g., corners, wall/floor boundaries, or semantic segmentation), or method of fitting the 3D layout. To address this challenge, we summarize and describe the common framework, the variants, and the impact of the design decisions. For a complete evaluation, we also propose extended annotations for the Matterport3D dataset (Chang et al.: Matterport3d: learning from rgb-d data in indoor environments. arXiv:1709.06158, 2017), and introduce two depth-based evaluation metrics.

Original languageEnglish (US)
Pages (from-to)1410-1431
Number of pages22
JournalInternational Journal of Computer Vision
Volume129
Issue number5
DOIs
StatePublished - May 2021

Keywords

  • 3D room layout
  • Deep learning
  • Manhattan world
  • Single image 3D

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Manhattan Room Layout Reconstruction from a Single 360 <sup>∘</sup> Image: A Comparative Study of State-of-the-Art Methods'. Together they form a unique fingerprint.

Cite this