Surface Snapping Optimization Layer for Single Image Object Shape Reconstruction

Yuan Ting Hu, Alexander G. Schwing, Raymond A. Yeh

Research output: Contribution to journalConference articlepeer-review


Reconstructing the 3D shape of objects observed in a single image is a challenging task. Recent approaches rely on visual cues extracted from a given image learned from a deep net. In this work, we leverage recent advances in monocular scene understanding to incorporate an additional geometric cue of surface normals. For this, we proposed a novel optimization layer that encourages the face normals of the reconstructed shape to be aligned with estimated surface normals. We develop a computationally efficient conjugate-gradient-based method that avoids the computation of high-dimensional sparse matrices. We show this framework to achieve compelling shape reconstruction results on the challenging Pix3D and ShapeNet datasets.

Original languageEnglish (US)
Pages (from-to)13599-13609
Number of pages11
JournalProceedings of Machine Learning Research
StatePublished - 2023
Externally publishedYes
Event40th International Conference on Machine Learning, ICML 2023 - Honolulu, United States
Duration: Jul 23 2023Jul 29 2023

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability


Dive into the research topics of 'Surface Snapping Optimization Layer for Single Image Object Shape Reconstruction'. Together they form a unique fingerprint.

Cite this