Colorful image colorization

Richard Yi Zhang, Phillip Isola, Alexei A. Efros

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Given a grayscale photograph as input, this paper attacks the problem of hallucinating a plausible color version of the photograph. This problem is clearly underconstrained, so previous approaches have either relied on significant user interaction or resulted in desaturated colorizations. We propose a fully automatic approach that produces vibrant and realistic colorizations. We embrace the underlying uncertainty of the problem by posing it as a classification task and use class-rebalancing at training time to increase the diversity of colors in the result. The system is implemented as a feed-forward pass in a CNN at test time and is trained on over a million color images. We evaluate our algorithm using a “colorization Turing test,” asking human participants to choose between a generated and ground truth color image. Our method successfully fools humans on 32% of the trials, significantly higher than previous methods. Moreover, we show that colorization can be a powerful pretext task for self-supervised feature learning, acting as a cross-channel encoder. This approach results in state-of-the-art performance on several feature learning benchmarks.

Original languageEnglish (US)
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
EditorsJiri Matas, Nicu Sebe, Max Welling, Bastian Leibe
PublisherSpringer
Pages649-666
Number of pages18
ISBN (Print)9783319464862
DOIs
StatePublished - Jan 1 2016
Externally publishedYes

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9907 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords

  • CNNs
  • Colorization
  • Self-supervised learning
  • Vision for graphics

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Colorful image colorization'. Together they form a unique fingerprint.

Cite this