On the Impact of Perceptual Compression on Deep Learning

Gerald Friedland, Rouxi Jia, Jingkang Wang, Bo Li, Nathan Mundhenk

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper proposes a fundamental answer to a frequently asked question in multimedia evaluation and data set creation: Do artifacts from perceptual compression contribute to error in the machine learning process and if so, how much? Our approach to the problem is an information reinterpretation of the Helmholtz free energy formula to explain the relationship between content and noise when using sensors (such as cameras or microphones) to capture multimedia data. The reinterpretation guides a bit-measurement of the noise contained in images, audio, and video by combining a classifier with perceptual compression, such as JPEG or MP3. Our experiments on CIFAR-10, ImageNet, and CSAIL Places as well as Fraunhofer's IDMT-SMT-Audio-Effects dataset indicate that, at the right quality level, perceptual compression is actually not harmful but contributes to a significant reduction of complexity of the machine learning process. That is, our noise quantification method can be used to speed up the training of deep learning classifiers significantly while maintaining, or sometimes even improving, overall classification accuracy.

Original languageEnglish (US)
Title of host publicationProceedings - 3rd International Conference on Multimedia Information Processing and Retrieval, MIPR 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages219-224
Number of pages6
ISBN (Electronic)9781728142722
DOIs
StatePublished - Aug 2020
Event3rd International Conference on Multimedia Information Processing and Retrieval, MIPR 2020 - Shenzhen, Guangdong, China
Duration: Aug 6 2020Aug 8 2020

Publication series

NameProceedings - 3rd International Conference on Multimedia Information Processing and Retrieval, MIPR 2020

Conference

Conference3rd International Conference on Multimedia Information Processing and Retrieval, MIPR 2020
Country/TerritoryChina
CityShenzhen, Guangdong
Period8/6/208/8/20

Keywords

  • Audio
  • Compression
  • Deep learning
  • Images
  • Information
  • Neural Networks
  • datasets

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Signal Processing
  • Media Technology
  • Modeling and Simulation
  • Library and Information Sciences

Fingerprint

Dive into the research topics of 'On the Impact of Perceptual Compression on Deep Learning'. Together they form a unique fingerprint.

Cite this