Joint image filtering with deep convolutional networks

Yijun Li, Jia Bin Huang, Narendra Ahuja, Ming Hsuan Yang

Research output: Contribution to journalArticle

Abstract

Joint image filters leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods either rely on various explicit filter constructions or hand-designed objective functions, thereby making it difficult to understand, improve, and accelerate these filters in a coherent framework. In this paper, we propose a learning-based approach for constructing joint filters based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, the proposed algorithm can selectively transfer salient structures that are consistent with both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well to other modalities, e.g., flash/non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive experimental evaluations with state-of-The-Art methods.

Original languageEnglish (US)
Article number8598855
Pages (from-to)1909-1923
Number of pages15
JournalIEEE transactions on pattern analysis and machine intelligence
Volume41
Issue number8
DOIs
StatePublished - Aug 1 2019

Fingerprint

Image Filtering
Neural networks
Guidance
Filter
Target
Flash
Experimental Evaluation
Leverage
Spatial Resolution
Modality
Accelerate
Objective function
Neural Networks
Generalise

Keywords

  • deep convolutional neural networks
  • depth upsampling
  • Joint filtering

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Cite this

Joint image filtering with deep convolutional networks. / Li, Yijun; Huang, Jia Bin; Ahuja, Narendra; Yang, Ming Hsuan.

In: IEEE transactions on pattern analysis and machine intelligence, Vol. 41, No. 8, 8598855, 01.08.2019, p. 1909-1923.

Research output: Contribution to journalArticle

Li, Yijun ; Huang, Jia Bin ; Ahuja, Narendra ; Yang, Ming Hsuan. / Joint image filtering with deep convolutional networks. In: IEEE transactions on pattern analysis and machine intelligence. 2019 ; Vol. 41, No. 8. pp. 1909-1923.
@article{f98881c330504817893277663c859e21,
title = "Joint image filtering with deep convolutional networks",
abstract = "Joint image filters leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods either rely on various explicit filter constructions or hand-designed objective functions, thereby making it difficult to understand, improve, and accelerate these filters in a coherent framework. In this paper, we propose a learning-based approach for constructing joint filters based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, the proposed algorithm can selectively transfer salient structures that are consistent with both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well to other modalities, e.g., flash/non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive experimental evaluations with state-of-The-Art methods.",
keywords = "deep convolutional neural networks, depth upsampling, Joint filtering",
author = "Yijun Li and Huang, {Jia Bin} and Narendra Ahuja and Yang, {Ming Hsuan}",
year = "2019",
month = "8",
day = "1",
doi = "10.1109/TPAMI.2018.2890623",
language = "English (US)",
volume = "41",
pages = "1909--1923",
journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
issn = "0162-8828",
publisher = "IEEE Computer Society",
number = "8",

}

TY - JOUR

T1 - Joint image filtering with deep convolutional networks

AU - Li, Yijun

AU - Huang, Jia Bin

AU - Ahuja, Narendra

AU - Yang, Ming Hsuan

PY - 2019/8/1

Y1 - 2019/8/1

N2 - Joint image filters leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods either rely on various explicit filter constructions or hand-designed objective functions, thereby making it difficult to understand, improve, and accelerate these filters in a coherent framework. In this paper, we propose a learning-based approach for constructing joint filters based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, the proposed algorithm can selectively transfer salient structures that are consistent with both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well to other modalities, e.g., flash/non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive experimental evaluations with state-of-The-Art methods.

AB - Joint image filters leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods either rely on various explicit filter constructions or hand-designed objective functions, thereby making it difficult to understand, improve, and accelerate these filters in a coherent framework. In this paper, we propose a learning-based approach for constructing joint filters based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, the proposed algorithm can selectively transfer salient structures that are consistent with both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well to other modalities, e.g., flash/non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive experimental evaluations with state-of-The-Art methods.

KW - deep convolutional neural networks

KW - depth upsampling

KW - Joint filtering

UR - http://www.scopus.com/inward/record.url?scp=85068881360&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068881360&partnerID=8YFLogxK

U2 - 10.1109/TPAMI.2018.2890623

DO - 10.1109/TPAMI.2018.2890623

M3 - Article

C2 - 30605094

AN - SCOPUS:85068881360

VL - 41

SP - 1909

EP - 1923

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

SN - 0162-8828

IS - 8

M1 - 8598855

ER -