Coreset-based neural network compression

Abhimanyu Dubey, Moitreya Chatterjee, Narendra Ahuja

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a novel Convolutional Neural Network (CNN) compression algorithm based on coreset representations of filters. We exploit the redundancies extant in the space of CNN weights and neuronal activations (across samples) in order to obtain compression. Our method requires no retraining, is easy to implement, and obtains state-of-the-art compression performance across a wide variety of CNN architectures. Coupled with quantization and Huffman coding, we create networks that provide AlexNet-like accuracy, with a memory footprint that is 832 × smaller than the original AlexNet, while also introducing significant reductions in inference time as well. Additionally these compressed networks when fine-tuned, successfully generalize to other domains as well.

Original languageEnglish (US)
Title of host publicationComputer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
EditorsVittorio Ferrari, Cristian Sminchisescu, Martial Hebert, Yair Weiss
PublisherSpringer-Verlag
Pages469-486
Number of pages18
ISBN (Print)9783030012335
DOIs
StatePublished - Jan 1 2018
Event15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany
Duration: Sep 8 2018Sep 14 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11211 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other15th European Conference on Computer Vision, ECCV 2018
CountryGermany
CityMunich
Period9/8/189/14/18

Fingerprint

Compression
Neural Networks
Neural networks
Huffman Coding
Network Architecture
Network architecture
Redundancy
Activation
Quantization
Chemical activation
Filter
Data storage equipment
Generalise

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Dubey, A., Chatterjee, M., & Ahuja, N. (2018). Coreset-based neural network compression. In V. Ferrari, C. Sminchisescu, M. Hebert, & Y. Weiss (Eds.), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings (pp. 469-486). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11211 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-01234-2_28

Coreset-based neural network compression. / Dubey, Abhimanyu; Chatterjee, Moitreya; Ahuja, Narendra.

Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Vittorio Ferrari; Cristian Sminchisescu; Martial Hebert; Yair Weiss. Springer-Verlag, 2018. p. 469-486 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11211 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Dubey, A, Chatterjee, M & Ahuja, N 2018, Coreset-based neural network compression. in V Ferrari, C Sminchisescu, M Hebert & Y Weiss (eds), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11211 LNCS, Springer-Verlag, pp. 469-486, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 9/8/18. https://doi.org/10.1007/978-3-030-01234-2_28
Dubey A, Chatterjee M, Ahuja N. Coreset-based neural network compression. In Ferrari V, Sminchisescu C, Hebert M, Weiss Y, editors, Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Springer-Verlag. 2018. p. 469-486. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-01234-2_28
Dubey, Abhimanyu ; Chatterjee, Moitreya ; Ahuja, Narendra. / Coreset-based neural network compression. Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. editor / Vittorio Ferrari ; Cristian Sminchisescu ; Martial Hebert ; Yair Weiss. Springer-Verlag, 2018. pp. 469-486 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{92083a223392410e800d23628ca1b746,
title = "Coreset-based neural network compression",
abstract = "We propose a novel Convolutional Neural Network (CNN) compression algorithm based on coreset representations of filters. We exploit the redundancies extant in the space of CNN weights and neuronal activations (across samples) in order to obtain compression. Our method requires no retraining, is easy to implement, and obtains state-of-the-art compression performance across a wide variety of CNN architectures. Coupled with quantization and Huffman coding, we create networks that provide AlexNet-like accuracy, with a memory footprint that is 832 × smaller than the original AlexNet, while also introducing significant reductions in inference time as well. Additionally these compressed networks when fine-tuned, successfully generalize to other domains as well.",
author = "Abhimanyu Dubey and Moitreya Chatterjee and Narendra Ahuja",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-030-01234-2_28",
language = "English (US)",
isbn = "9783030012335",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer-Verlag",
pages = "469--486",
editor = "Vittorio Ferrari and Cristian Sminchisescu and Martial Hebert and Yair Weiss",
booktitle = "Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings",

}

TY - GEN

T1 - Coreset-based neural network compression

AU - Dubey, Abhimanyu

AU - Chatterjee, Moitreya

AU - Ahuja, Narendra

PY - 2018/1/1

Y1 - 2018/1/1

N2 - We propose a novel Convolutional Neural Network (CNN) compression algorithm based on coreset representations of filters. We exploit the redundancies extant in the space of CNN weights and neuronal activations (across samples) in order to obtain compression. Our method requires no retraining, is easy to implement, and obtains state-of-the-art compression performance across a wide variety of CNN architectures. Coupled with quantization and Huffman coding, we create networks that provide AlexNet-like accuracy, with a memory footprint that is 832 × smaller than the original AlexNet, while also introducing significant reductions in inference time as well. Additionally these compressed networks when fine-tuned, successfully generalize to other domains as well.

AB - We propose a novel Convolutional Neural Network (CNN) compression algorithm based on coreset representations of filters. We exploit the redundancies extant in the space of CNN weights and neuronal activations (across samples) in order to obtain compression. Our method requires no retraining, is easy to implement, and obtains state-of-the-art compression performance across a wide variety of CNN architectures. Coupled with quantization and Huffman coding, we create networks that provide AlexNet-like accuracy, with a memory footprint that is 832 × smaller than the original AlexNet, while also introducing significant reductions in inference time as well. Additionally these compressed networks when fine-tuned, successfully generalize to other domains as well.

UR - http://www.scopus.com/inward/record.url?scp=85055091388&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055091388&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01234-2_28

DO - 10.1007/978-3-030-01234-2_28

M3 - Conference contribution

AN - SCOPUS:85055091388

SN - 9783030012335

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 469

EP - 486

BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings

A2 - Ferrari, Vittorio

A2 - Sminchisescu, Cristian

A2 - Hebert, Martial

A2 - Weiss, Yair

PB - Springer-Verlag

ER -