Towards Accurate 3D Human Body Reconstruction from Silhouettes

Brandon M. Smith, Visesh Chari, Amit Agrawal, James M. Rehg, Ram Sever

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We propose a novel computer vision system for reconstructing 3D body shapes from 2D images with the goal of producing highly accurate anthropomorphic measurements from a pair of images. We adopt a supervised learning approach that maps silhouette images to 3D body shapes via a convolutional neural network (CNN). We propose three key improvements over previous approaches: (1) Large-scale realistic synthetic data generation, including more realistic variations in segmentation noise and camera viewpoints. (2) A multi-task learning (MTL) approach to predicting multiple outputs such as shape, 3D joint locations, pose angles, and body volume. (3) A new network architecture that additionally takes known body measurements (e.g., height) and per-pixel segmentation confidence as input. Ablation studies show the improvement in accuracy due to the various components of our system. Results demonstrate that our system produces state-of-the-art results on body circumference errors. We also analyze the repeatability of our system in the presence of realistic camera, background, and pose variations. Our system achieves a vertex standard deviation of ~3mm on the [36] CAESAR dataset.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 International Conference on 3D Vision, 3DV 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages10
ISBN (Electronic)9781728131313
StatePublished - Sep 2019
Externally publishedYes
Event7th International Conference on 3D Vision, 3DV 2019 - Quebec, Canada
Duration: Sep 15 2019Sep 18 2019

Publication series

NameProceedings - 2019 International Conference on 3D Vision, 3DV 2019


Conference7th International Conference on 3D Vision, 3DV 2019


  • Anthropomorphic Measurements
  • Human Body Reconstruction
  • Multitask Learning
  • Segmentation Confidence
  • Shape From Silhouette
  • Synthetic Data

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Media Technology
  • Modeling and Simulation


Dive into the research topics of 'Towards Accurate 3D Human Body Reconstruction from Silhouettes'. Together they form a unique fingerprint.

Cite this