VecQ: Minimal Loss DNN Model Compression with Vectorized Weight Quantization

Cheng Gong, Yao Chen, Ye Lu, Tao Li, Cong Hao, Deming Chen

Research output: Contribution to journalArticlepeer-review


Quantization has been proven to be an effective method for reducing the computing and/or storage cost of DNNs. However, the trade-off between the quantization bitwidth and final accuracy is complex and non-convex, which makes it difficult to be optimized directly. Minimizing direct quantization loss (DQL) of the coefficient data is an effective local optimization method, but previous works often neglect the accurate control of the DQL, resulting in a higher loss of the final DNN model accuracy. In this paper, we propose a novel metric, called Vector Loss. Using this new metric, we decompose the minimization of the DQL to two independent optimization processes, which significantly outperform the traditional iterative L2 loss minimization process in terms of effectiveness, quantization loss as well as final DNN accuracy. We also develop a new DNN quantization solution called VecQ, which provides minimal direct quantization loss and achieve higher model accuracy. In order to speed up the proposed quantization process during model training, we accelerate the quantization process with a parameterized probability estimation method and template-based derivation calculation. We evaluate our proposed algorithm on MNIST, CIFAR, ImageNet, IMDB movie review and THUCNews text data sets with numerical DNN models. The results demonstrate that our proposed quantization solution is more accurate and effective than the state-of-the-art approaches yet with more flexible bitwidth support. Moreover, the evaluation of our quantized models on Salient Object Detection (SOD) tasks maintains comparable feature extraction quality with up to 16× weight size reduction.

Original languageEnglish (US)
Article number9095420
Pages (from-to)696-710
Number of pages15
JournalIEEE Transactions on Computers
Issue number5
StatePublished - May 1 2021
Externally publishedYes


  • DNN compression
  • DNN quantization
  • low bitwidth
  • vector loss
  • vectorized weight quantization

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture
  • Computational Theory and Mathematics


Dive into the research topics of 'VecQ: Minimal Loss DNN Model Compression with Vectorized Weight Quantization'. Together they form a unique fingerprint.

Cite this