Abstract
Colorization is an ambiguous problem, with multiple viable colorizations for a single grey-level image. However, previous methods only produce the single most probable colorization. Our goal is to model the diversity intrinsic to the problem of colorization and produce multiple colorizations that display long-scale spatial co-ordination. We learn a low dimensional embedding of color fields using a variational autoencoder (VAE). We construct loss terms for the VAE decoder that avoid blurry outputs and take into account the uneven distribution of pixel colors. Finally, we build a conditional model for the multi-modal distribution between grey-level image and the color field embeddings. Samples from this conditional model result in diverse colorization. We demonstrate that our method obtains better diverse colorizations than a standard conditional variational autoencoder (CVAE) model, as well as a recently proposed conditional generative adversarial network (cGAN).
| Original language | English (US) |
|---|---|
| Title of host publication | Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 2877-2885 |
| Number of pages | 9 |
| ISBN (Electronic) | 9781538604571 |
| DOIs | |
| State | Published - Nov 6 2017 |
| Event | 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 - Honolulu, United States Duration: Jul 21 2017 → Jul 26 2017 |
Publication series
| Name | Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 |
|---|---|
| Volume | 2017-January |
Other
| Other | 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 |
|---|---|
| Country | United States |
| City | Honolulu |
| Period | 7/21/17 → 7/26/17 |
Fingerprint
ASJC Scopus subject areas
- Signal Processing
- Computer Vision and Pattern Recognition
Cite this
Learning diverse image colorization. / Deshpande, Aditya; Lu, Jiajun; Yeh, Mao Chuang; Chong, Min Jin; Forsyth, David Alexander.
Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 2877-2885 (Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017; Vol. 2017-January).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Learning diverse image colorization
AU - Deshpande, Aditya
AU - Lu, Jiajun
AU - Yeh, Mao Chuang
AU - Chong, Min Jin
AU - Forsyth, David Alexander
PY - 2017/11/6
Y1 - 2017/11/6
N2 - Colorization is an ambiguous problem, with multiple viable colorizations for a single grey-level image. However, previous methods only produce the single most probable colorization. Our goal is to model the diversity intrinsic to the problem of colorization and produce multiple colorizations that display long-scale spatial co-ordination. We learn a low dimensional embedding of color fields using a variational autoencoder (VAE). We construct loss terms for the VAE decoder that avoid blurry outputs and take into account the uneven distribution of pixel colors. Finally, we build a conditional model for the multi-modal distribution between grey-level image and the color field embeddings. Samples from this conditional model result in diverse colorization. We demonstrate that our method obtains better diverse colorizations than a standard conditional variational autoencoder (CVAE) model, as well as a recently proposed conditional generative adversarial network (cGAN).
AB - Colorization is an ambiguous problem, with multiple viable colorizations for a single grey-level image. However, previous methods only produce the single most probable colorization. Our goal is to model the diversity intrinsic to the problem of colorization and produce multiple colorizations that display long-scale spatial co-ordination. We learn a low dimensional embedding of color fields using a variational autoencoder (VAE). We construct loss terms for the VAE decoder that avoid blurry outputs and take into account the uneven distribution of pixel colors. Finally, we build a conditional model for the multi-modal distribution between grey-level image and the color field embeddings. Samples from this conditional model result in diverse colorization. We demonstrate that our method obtains better diverse colorizations than a standard conditional variational autoencoder (CVAE) model, as well as a recently proposed conditional generative adversarial network (cGAN).
UR - http://www.scopus.com/inward/record.url?scp=85044377671&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85044377671&partnerID=8YFLogxK
U2 - 10.1109/CVPR.2017.307
DO - 10.1109/CVPR.2017.307
M3 - Conference contribution
AN - SCOPUS:85044377671
T3 - Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
SP - 2877
EP - 2885
BT - Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
PB - Institute of Electrical and Electronics Engineers Inc.
ER -