Deep joint image filtering

Yijun Li, Jia Bin Huang, Narendra Ahuja, Ming Hsuan Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Joint image filters can leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods rely on various kinds of explicit filter construction or handdesigned objective functions. It is thus difficult to understand, improve, and accelerate them in a coherent framework. In this paper, we propose a learning-based approach to construct a joint filter based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, our method can selectively transfer salient structures that are consistent in both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well for other modalities, e.g., Flash/Non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.

Original languageEnglish (US)
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
EditorsNicu Sebe, Bastian Leibe, Max Welling, Jiri Matas
PublisherSpringer-Verlag
Pages154-169
Number of pages16
ISBN (Print)9783319464923
DOIs
StatePublished - Jan 1 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9908 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Fingerprint

Image Filtering
Neural networks
Guidance
Filter
Target
Flash
Leverage
Spatial Resolution
Modality
Accelerate
Objective function
Neural Networks
Generalise

Keywords

  • Deep convolutional neural networks
  • Joint filtering

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Li, Y., Huang, J. B., Ahuja, N., & Yang, M. H. (2016). Deep joint image filtering. In N. Sebe, B. Leibe, M. Welling, & J. Matas (Eds.), Computer Vision - 14th European Conference, ECCV 2016, Proceedings (pp. 154-169). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9908 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-319-46493-0_10

Deep joint image filtering. / Li, Yijun; Huang, Jia Bin; Ahuja, Narendra; Yang, Ming Hsuan.

Computer Vision - 14th European Conference, ECCV 2016, Proceedings. ed. / Nicu Sebe; Bastian Leibe; Max Welling; Jiri Matas. Springer-Verlag, 2016. p. 154-169 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9908 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Li, Y, Huang, JB, Ahuja, N & Yang, MH 2016, Deep joint image filtering. in N Sebe, B Leibe, M Welling & J Matas (eds), Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9908 LNCS, Springer-Verlag, pp. 154-169. https://doi.org/10.1007/978-3-319-46493-0_10
Li Y, Huang JB, Ahuja N, Yang MH. Deep joint image filtering. In Sebe N, Leibe B, Welling M, Matas J, editors, Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Springer-Verlag. 2016. p. 154-169. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46493-0_10
Li, Yijun ; Huang, Jia Bin ; Ahuja, Narendra ; Yang, Ming Hsuan. / Deep joint image filtering. Computer Vision - 14th European Conference, ECCV 2016, Proceedings. editor / Nicu Sebe ; Bastian Leibe ; Max Welling ; Jiri Matas. Springer-Verlag, 2016. pp. 154-169 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{1af7b799577c4d8cab4c8c4cf0fb79c9,
title = "Deep joint image filtering",
abstract = "Joint image filters can leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods rely on various kinds of explicit filter construction or handdesigned objective functions. It is thus difficult to understand, improve, and accelerate them in a coherent framework. In this paper, we propose a learning-based approach to construct a joint filter based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, our method can selectively transfer salient structures that are consistent in both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well for other modalities, e.g., Flash/Non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.",
keywords = "Deep convolutional neural networks, Joint filtering",
author = "Yijun Li and Huang, {Jia Bin} and Narendra Ahuja and Yang, {Ming Hsuan}",
year = "2016",
month = "1",
day = "1",
doi = "10.1007/978-3-319-46493-0_10",
language = "English (US)",
isbn = "9783319464923",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer-Verlag",
pages = "154--169",
editor = "Nicu Sebe and Bastian Leibe and Max Welling and Jiri Matas",
booktitle = "Computer Vision - 14th European Conference, ECCV 2016, Proceedings",

}

TY - GEN

T1 - Deep joint image filtering

AU - Li, Yijun

AU - Huang, Jia Bin

AU - Ahuja, Narendra

AU - Yang, Ming Hsuan

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Joint image filters can leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods rely on various kinds of explicit filter construction or handdesigned objective functions. It is thus difficult to understand, improve, and accelerate them in a coherent framework. In this paper, we propose a learning-based approach to construct a joint filter based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, our method can selectively transfer salient structures that are consistent in both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well for other modalities, e.g., Flash/Non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.

AB - Joint image filters can leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods rely on various kinds of explicit filter construction or handdesigned objective functions. It is thus difficult to understand, improve, and accelerate them in a coherent framework. In this paper, we propose a learning-based approach to construct a joint filter based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, our method can selectively transfer salient structures that are consistent in both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well for other modalities, e.g., Flash/Non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.

KW - Deep convolutional neural networks

KW - Joint filtering

UR - http://www.scopus.com/inward/record.url?scp=84990053231&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990053231&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46493-0_10

DO - 10.1007/978-3-319-46493-0_10

M3 - Conference contribution

AN - SCOPUS:84990053231

SN - 9783319464923

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 154

EP - 169

BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings

A2 - Sebe, Nicu

A2 - Leibe, Bastian

A2 - Welling, Max

A2 - Matas, Jiri

PB - Springer-Verlag

ER -