Abstract
Colorizing a given gray-level image is an important task in the media and advertising industry. Due to the ambiguity inherent to colorization (many shades are often plausible), recent approaches started to explicitly model diversity. However, one of the most obvious artifacts, structural inconsistency, is rarely considered by existing methods which predict chrominance independently for every pixel. To address this issue, we develop a conditional random field based variational auto-encoder formulation which is able to achieve diversity while taking into account structural consistency. Moreover, we introduce a controllability mechanism that can incorporate external constraints from diverse sources including a user interface. Compared to existing baselines, we demonstrate that our method obtains more diverse and globally consistent colorizations on the LFW, LSUN-Church and ILSVRC-2015 datasets.
| Original language | English (US) |
|---|---|
| Title of host publication | Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings |
| Editors | Martial Hebert, Yair Weiss, Vittorio Ferrari, Cristian Sminchisescu |
| Publisher | Springer-Verlag Berlin Heidelberg |
| Pages | 603-619 |
| Number of pages | 17 |
| ISBN (Print) | 9783030012304 |
| DOIs | |
| State | Published - Jan 1 2018 |
| Event | 15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: Sep 8 2018 → Sep 14 2018 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 11210 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Other
| Other | 15th European Conference on Computer Vision, ECCV 2018 |
|---|---|
| Country | Germany |
| City | Munich |
| Period | 9/8/18 → 9/14/18 |
Fingerprint
Keywords
- Colorization
- Gaussian-Conditional Random Field
- VAE
ASJC Scopus subject areas
- Theoretical Computer Science
- Computer Science(all)
Cite this
Structural consistency and controllability for diverse colorization. / Messaoud, Safa; Forsyth, David; Schwing, Alexander G.
Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Martial Hebert; Yair Weiss; Vittorio Ferrari; Cristian Sminchisescu. Springer-Verlag Berlin Heidelberg, 2018. p. 603-619 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11210 LNCS).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Structural consistency and controllability for diverse colorization
AU - Messaoud, Safa
AU - Forsyth, David
AU - Schwing, Alexander G.
PY - 2018/1/1
Y1 - 2018/1/1
N2 - Colorizing a given gray-level image is an important task in the media and advertising industry. Due to the ambiguity inherent to colorization (many shades are often plausible), recent approaches started to explicitly model diversity. However, one of the most obvious artifacts, structural inconsistency, is rarely considered by existing methods which predict chrominance independently for every pixel. To address this issue, we develop a conditional random field based variational auto-encoder formulation which is able to achieve diversity while taking into account structural consistency. Moreover, we introduce a controllability mechanism that can incorporate external constraints from diverse sources including a user interface. Compared to existing baselines, we demonstrate that our method obtains more diverse and globally consistent colorizations on the LFW, LSUN-Church and ILSVRC-2015 datasets.
AB - Colorizing a given gray-level image is an important task in the media and advertising industry. Due to the ambiguity inherent to colorization (many shades are often plausible), recent approaches started to explicitly model diversity. However, one of the most obvious artifacts, structural inconsistency, is rarely considered by existing methods which predict chrominance independently for every pixel. To address this issue, we develop a conditional random field based variational auto-encoder formulation which is able to achieve diversity while taking into account structural consistency. Moreover, we introduce a controllability mechanism that can incorporate external constraints from diverse sources including a user interface. Compared to existing baselines, we demonstrate that our method obtains more diverse and globally consistent colorizations on the LFW, LSUN-Church and ILSVRC-2015 datasets.
KW - Colorization
KW - Gaussian-Conditional Random Field
KW - VAE
UR - http://www.scopus.com/inward/record.url?scp=85055088417&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85055088417&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-01231-1_37
DO - 10.1007/978-3-030-01231-1_37
M3 - Conference contribution
AN - SCOPUS:85055088417
SN - 9783030012304
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 603
EP - 619
BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
A2 - Hebert, Martial
A2 - Weiss, Yair
A2 - Ferrari, Vittorio
A2 - Sminchisescu, Cristian
PB - Springer-Verlag Berlin Heidelberg
ER -