3D-Filtermap: A compact architecture for deep convolutional neural networks

Yingzhen Yang, Jianchao Yang, Ning Xu, Wei Han, Nebojsa Jojic, Thomas S. Huang

Research output: Contribution to conferencePaper

Abstract

We present a novel and compact architecture for deep Convolutional Neural Networks (CNNs) in this paper, termed 3D-FilterMap Convolutional Neural Networks (3D-FM-CNNs). The convolution layer of 3D-FM-CNN learns a compact representation of the filters, named 3D-FilterMap, instead of a set of independent filters in the conventional convolution layer. The filters are extracted from the 3D-FilterMap as overlapping 3D submatrics with weight sharing among nearby filters, and these filters are convolved with the input to generate the output of the convolution layer for 3D-FM-CNN. Due to the weight sharing scheme, the parameter size of the 3D-FilterMap is much smaller than that of the filters to be learned in the conventional convolution layer when 3D-FilterMap generates the same number of filters. Our work is fundamentally different from the network compression literature that reduces the size of a learned large network in the sense that a small network is directly learned from scratch. Experimental results demonstrate that 3D-FM-CNN enjoys a small parameter space by learning compact 3D-FilterMaps, while achieving performance compared to that of the baseline CNNs which learn the same number of filters as that generated by the corresponding 3D-FilterMap.

Original languageEnglish (US)
StatePublished - Jan 1 2018
Event6th International Conference on Learning Representations, ICLR 2018 - Vancouver, Canada
Duration: Apr 30 2018May 3 2018

Conference

Conference6th International Conference on Learning Representations, ICLR 2018
CountryCanada
CityVancouver
Period4/30/185/3/18

ASJC Scopus subject areas

  • Education
  • Computer Science Applications
  • Linguistics and Language
  • Language and Linguistics

Fingerprint Dive into the research topics of '3D-Filtermap: A compact architecture for deep convolutional neural networks'. Together they form a unique fingerprint.

  • Cite this

    Yang, Y., Yang, J., Xu, N., Han, W., Jojic, N., & Huang, T. S. (2018). 3D-Filtermap: A compact architecture for deep convolutional neural networks. Paper presented at 6th International Conference on Learning Representations, ICLR 2018, Vancouver, Canada.