Blind recovery of spatially varying reflectance from a single image

Kevin Karsch, David Forsyth

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a new technique for estimating spatially varying parametric materials from a single image of an object with unknown shape in unknown illumination. Our method uses a low-order parametric reflectance model, and incorporates strong assumptions about lighting and shape. We develop new priors about how materials mix over space, and jointly infer all of these properties from a single image. This produces a decomposition of an image which corresponds, in one sense, to microscopic features (material reflectance) and macroscopic features (weights defining the mixing properties of materials over space). We have built a large dataset of real objects rendered with different material models under different illumination fields for training and ground truth evaluation. Extensive experiments on both our synthetic dataset images as well as real images show that (a) our method recovers parameters with reasonable accuracy; (b) material parameters recovered by our method give accurate predictions of new renderings of the object; and (c) our low-order reflectance model still provides a good fit to many real-world reflectances.

Original languageEnglish (US)
Title of host publicationSIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450332422
DOIs
StatePublished - Nov 24 2014
EventSIGGRAPH Asia 2014 Workshop on Indoor Scene Understanding Where Graphics Meets Vision, SA 2014 - Shenzhen, China
Duration: Dec 3 2014Dec 6 2014

Publication series

NameSIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014

Other

OtherSIGGRAPH Asia 2014 Workshop on Indoor Scene Understanding Where Graphics Meets Vision, SA 2014
CountryChina
CityShenzhen
Period12/3/1412/6/14

Fingerprint

Recovery
Lighting
Decomposition
Experiments

Keywords

  • Material modeling
  • Material transfer
  • Reflectance estimation
  • Shape from shading

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Karsch, K., & Forsyth, D. (2014). Blind recovery of spatially varying reflectance from a single image. In SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014 [2] (SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014). Association for Computing Machinery, Inc. https://doi.org/10.1145/2670291.2670293

Blind recovery of spatially varying reflectance from a single image. / Karsch, Kevin; Forsyth, David.

SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014. Association for Computing Machinery, Inc, 2014. 2 (SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Karsch, K & Forsyth, D 2014, Blind recovery of spatially varying reflectance from a single image. in SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014., 2, SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014, Association for Computing Machinery, Inc, SIGGRAPH Asia 2014 Workshop on Indoor Scene Understanding Where Graphics Meets Vision, SA 2014, Shenzhen, China, 12/3/14. https://doi.org/10.1145/2670291.2670293
Karsch K, Forsyth D. Blind recovery of spatially varying reflectance from a single image. In SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014. Association for Computing Machinery, Inc. 2014. 2. (SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014). https://doi.org/10.1145/2670291.2670293
Karsch, Kevin ; Forsyth, David. / Blind recovery of spatially varying reflectance from a single image. SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014. Association for Computing Machinery, Inc, 2014. (SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014).
@inproceedings{45b685b2c72245c085768cdae117d266,
title = "Blind recovery of spatially varying reflectance from a single image",
abstract = "We propose a new technique for estimating spatially varying parametric materials from a single image of an object with unknown shape in unknown illumination. Our method uses a low-order parametric reflectance model, and incorporates strong assumptions about lighting and shape. We develop new priors about how materials mix over space, and jointly infer all of these properties from a single image. This produces a decomposition of an image which corresponds, in one sense, to microscopic features (material reflectance) and macroscopic features (weights defining the mixing properties of materials over space). We have built a large dataset of real objects rendered with different material models under different illumination fields for training and ground truth evaluation. Extensive experiments on both our synthetic dataset images as well as real images show that (a) our method recovers parameters with reasonable accuracy; (b) material parameters recovered by our method give accurate predictions of new renderings of the object; and (c) our low-order reflectance model still provides a good fit to many real-world reflectances.",
keywords = "Material modeling, Material transfer, Reflectance estimation, Shape from shading",
author = "Kevin Karsch and David Forsyth",
year = "2014",
month = "11",
day = "24",
doi = "10.1145/2670291.2670293",
language = "English (US)",
series = "SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014",
publisher = "Association for Computing Machinery, Inc",
booktitle = "SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014",

}

TY - GEN

T1 - Blind recovery of spatially varying reflectance from a single image

AU - Karsch, Kevin

AU - Forsyth, David

PY - 2014/11/24

Y1 - 2014/11/24

N2 - We propose a new technique for estimating spatially varying parametric materials from a single image of an object with unknown shape in unknown illumination. Our method uses a low-order parametric reflectance model, and incorporates strong assumptions about lighting and shape. We develop new priors about how materials mix over space, and jointly infer all of these properties from a single image. This produces a decomposition of an image which corresponds, in one sense, to microscopic features (material reflectance) and macroscopic features (weights defining the mixing properties of materials over space). We have built a large dataset of real objects rendered with different material models under different illumination fields for training and ground truth evaluation. Extensive experiments on both our synthetic dataset images as well as real images show that (a) our method recovers parameters with reasonable accuracy; (b) material parameters recovered by our method give accurate predictions of new renderings of the object; and (c) our low-order reflectance model still provides a good fit to many real-world reflectances.

AB - We propose a new technique for estimating spatially varying parametric materials from a single image of an object with unknown shape in unknown illumination. Our method uses a low-order parametric reflectance model, and incorporates strong assumptions about lighting and shape. We develop new priors about how materials mix over space, and jointly infer all of these properties from a single image. This produces a decomposition of an image which corresponds, in one sense, to microscopic features (material reflectance) and macroscopic features (weights defining the mixing properties of materials over space). We have built a large dataset of real objects rendered with different material models under different illumination fields for training and ground truth evaluation. Extensive experiments on both our synthetic dataset images as well as real images show that (a) our method recovers parameters with reasonable accuracy; (b) material parameters recovered by our method give accurate predictions of new renderings of the object; and (c) our low-order reflectance model still provides a good fit to many real-world reflectances.

KW - Material modeling

KW - Material transfer

KW - Reflectance estimation

KW - Shape from shading

UR - http://www.scopus.com/inward/record.url?scp=84919389347&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84919389347&partnerID=8YFLogxK

U2 - 10.1145/2670291.2670293

DO - 10.1145/2670291.2670293

M3 - Conference contribution

AN - SCOPUS:84919389347

T3 - SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014

BT - SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, SA 2014

PB - Association for Computing Machinery, Inc

ER -