DeepMVS: Learning Multi-view Stereopsis

Po Han Huang, Kevin Matzen, Johannes Kopf, Narendra Ahuja, Jia Bin Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present DeepMVS, a deep convolutional neural network (ConvNet) for multi-view stereo reconstruction. Taking an arbitrary number of posed images as input, we first produce a set of plane-sweep volumes and use the proposed DeepMVS network to predict high-quality disparity maps. The key contributions that enable these results are (1) supervised pretraining on a photorealistic synthetic dataset, (2) an effective method for aggregating information across a set of unordered images, and (3) integrating multi-layer feature activations from the pre-trained VGG-19 network. We validate the efficacy of DeepMVS using the ETH3D Benchmark. Our results show that DeepMVS compares favorably against state-of-the-art conventional MVS algorithms and other ConvNet based methods, particularly for near-textureless regions and thin structures.

Original languageEnglish (US)
Title of host publicationProceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018
PublisherIEEE Computer Society
Pages2821-2830
Number of pages10
ISBN (Electronic)9781538664209
DOIs
StatePublished - Dec 14 2018
Event31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018 - Salt Lake City, United States
Duration: Jun 18 2018Jun 22 2018

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN (Print)1063-6919

Conference

Conference31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018
CountryUnited States
CitySalt Lake City
Period6/18/186/22/18

Fingerprint

Neural networks
Chemical activation

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Huang, P. H., Matzen, K., Kopf, J., Ahuja, N., & Huang, J. B. (2018). DeepMVS: Learning Multi-view Stereopsis. In Proceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018 (pp. 2821-2830). [8578396] (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition). IEEE Computer Society. https://doi.org/10.1109/CVPR.2018.00298

DeepMVS : Learning Multi-view Stereopsis. / Huang, Po Han; Matzen, Kevin; Kopf, Johannes; Ahuja, Narendra; Huang, Jia Bin.

Proceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018. IEEE Computer Society, 2018. p. 2821-2830 8578396 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Huang, PH, Matzen, K, Kopf, J, Ahuja, N & Huang, JB 2018, DeepMVS: Learning Multi-view Stereopsis. in Proceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018., 8578396, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, pp. 2821-2830, 31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, United States, 6/18/18. https://doi.org/10.1109/CVPR.2018.00298
Huang PH, Matzen K, Kopf J, Ahuja N, Huang JB. DeepMVS: Learning Multi-view Stereopsis. In Proceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018. IEEE Computer Society. 2018. p. 2821-2830. 8578396. (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition). https://doi.org/10.1109/CVPR.2018.00298
Huang, Po Han ; Matzen, Kevin ; Kopf, Johannes ; Ahuja, Narendra ; Huang, Jia Bin. / DeepMVS : Learning Multi-view Stereopsis. Proceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018. IEEE Computer Society, 2018. pp. 2821-2830 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition).
@inproceedings{a46a7ab3958a48b08f9f7ed486c1a2c3,
title = "DeepMVS: Learning Multi-view Stereopsis",
abstract = "We present DeepMVS, a deep convolutional neural network (ConvNet) for multi-view stereo reconstruction. Taking an arbitrary number of posed images as input, we first produce a set of plane-sweep volumes and use the proposed DeepMVS network to predict high-quality disparity maps. The key contributions that enable these results are (1) supervised pretraining on a photorealistic synthetic dataset, (2) an effective method for aggregating information across a set of unordered images, and (3) integrating multi-layer feature activations from the pre-trained VGG-19 network. We validate the efficacy of DeepMVS using the ETH3D Benchmark. Our results show that DeepMVS compares favorably against state-of-the-art conventional MVS algorithms and other ConvNet based methods, particularly for near-textureless regions and thin structures.",
author = "Huang, {Po Han} and Kevin Matzen and Johannes Kopf and Narendra Ahuja and Huang, {Jia Bin}",
year = "2018",
month = "12",
day = "14",
doi = "10.1109/CVPR.2018.00298",
language = "English (US)",
series = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",
publisher = "IEEE Computer Society",
pages = "2821--2830",
booktitle = "Proceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018",

}

TY - GEN

T1 - DeepMVS

T2 - Learning Multi-view Stereopsis

AU - Huang, Po Han

AU - Matzen, Kevin

AU - Kopf, Johannes

AU - Ahuja, Narendra

AU - Huang, Jia Bin

PY - 2018/12/14

Y1 - 2018/12/14

N2 - We present DeepMVS, a deep convolutional neural network (ConvNet) for multi-view stereo reconstruction. Taking an arbitrary number of posed images as input, we first produce a set of plane-sweep volumes and use the proposed DeepMVS network to predict high-quality disparity maps. The key contributions that enable these results are (1) supervised pretraining on a photorealistic synthetic dataset, (2) an effective method for aggregating information across a set of unordered images, and (3) integrating multi-layer feature activations from the pre-trained VGG-19 network. We validate the efficacy of DeepMVS using the ETH3D Benchmark. Our results show that DeepMVS compares favorably against state-of-the-art conventional MVS algorithms and other ConvNet based methods, particularly for near-textureless regions and thin structures.

AB - We present DeepMVS, a deep convolutional neural network (ConvNet) for multi-view stereo reconstruction. Taking an arbitrary number of posed images as input, we first produce a set of plane-sweep volumes and use the proposed DeepMVS network to predict high-quality disparity maps. The key contributions that enable these results are (1) supervised pretraining on a photorealistic synthetic dataset, (2) an effective method for aggregating information across a set of unordered images, and (3) integrating multi-layer feature activations from the pre-trained VGG-19 network. We validate the efficacy of DeepMVS using the ETH3D Benchmark. Our results show that DeepMVS compares favorably against state-of-the-art conventional MVS algorithms and other ConvNet based methods, particularly for near-textureless regions and thin structures.

UR - http://www.scopus.com/inward/record.url?scp=85062817058&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062817058&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2018.00298

DO - 10.1109/CVPR.2018.00298

M3 - Conference contribution

AN - SCOPUS:85062817058

T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SP - 2821

EP - 2830

BT - Proceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018

PB - IEEE Computer Society

ER -