TY - JOUR
T1 - Enhance Visual Recognition Under Adverse Conditions via Deep Networks
AU - Liu, Ding
AU - Cheng, Bowen
AU - Wang, Zhangyang
AU - Zhang, Haichao
AU - Huang, Thomas S.
N1 - Manuscript received December 20, 2017; revised July 31, 2018 and October 28, 2018; accepted March 27, 2019. Date of publication April 1, 2019; date of current version July 1, 2019. This work was supported in part by the U.S. Army Research Office under Grant W911NF-15-1-0317. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Gang Hua. (Ding Liu and Bowen Cheng contributed equally to this work.) (Corresponding author: Ding Liu.) D. Liu, B. Cheng, and T. S. Huang are with the Department of Electrical and Computer Engineering and Beckman Institute, University of Illinois at Urbana–Champaign, Urbana, IL 61801 USA (e-mail: [email protected]; [email protected]; [email protected]).
PY - 2019/9
Y1 - 2019/9
N2 - Visual recognition under adverse conditions is a very important and challenging problem of high practical value, due to the ubiquitous existence of quality distortions during image acquisition, transmission, or storage. While deep neural networks have been extensively exploited in the techniques of low-quality image restoration and high-quality image recognition tasks, respectively, few studies have been done on the important problem of recognition from very low-quality images. This paper proposes a deep learning-based framework for improving the performance of image and video recognition models under adverse conditions, using robust adverse pre-training or its aggressive variant. The robust adverse pre-training algorithms leverage the power of pre-training and generalize the conventional unsupervised pre-training and data augmentation methods. We further develop a transfer learning approach to cope with real-world datasets of unknown adverse conditions. The proposed framework is comprehensively evaluated on a number of image and video recognition benchmarks, and obtains significant performance improvements under various single or mixed adverse conditions. Our visualization and analysis further add to the explainability of the results.
AB - Visual recognition under adverse conditions is a very important and challenging problem of high practical value, due to the ubiquitous existence of quality distortions during image acquisition, transmission, or storage. While deep neural networks have been extensively exploited in the techniques of low-quality image restoration and high-quality image recognition tasks, respectively, few studies have been done on the important problem of recognition from very low-quality images. This paper proposes a deep learning-based framework for improving the performance of image and video recognition models under adverse conditions, using robust adverse pre-training or its aggressive variant. The robust adverse pre-training algorithms leverage the power of pre-training and generalize the conventional unsupervised pre-training and data augmentation methods. We further develop a transfer learning approach to cope with real-world datasets of unknown adverse conditions. The proposed framework is comprehensively evaluated on a number of image and video recognition benchmarks, and obtains significant performance improvements under various single or mixed adverse conditions. Our visualization and analysis further add to the explainability of the results.
KW - Deep learning
KW - image recognition
KW - neural network
UR - http://www.scopus.com/inward/record.url?scp=85068362489&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85068362489&partnerID=8YFLogxK
U2 - 10.1109/TIP.2019.2908802
DO - 10.1109/TIP.2019.2908802
M3 - Article
C2 - 30946668
AN - SCOPUS:85068362489
SN - 1057-7149
VL - 28
SP - 4401
EP - 4412
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
IS - 9
M1 - 8678738
ER -