Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators

Sitao Huang; Aayush Ankit; Plinio Silveira; Rodrigo Antunes; Sai Rahul Chalamalasetti; Izzat El Hajj; Dong Eun Kim; Glaucimar Aguiar; Pedro Bruel; Sergey Serebryakov; Cong Xu; Can Li; Paolo Faraboschi; John Paul Strachan; Deming Chen; Kaushik Roy; Wen Mei Hwu; Dejan Milojicic

doi:10.1145/3394885.3431554

Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators

Sitao Huang, Aayush Ankit, Plinio Silveira, Rodrigo Antunes, Sai Rahul Chalamalasetti, Izzat El Hajj, Dong Eun Kim, Glaucimar Aguiar, Pedro Bruel, Sergey Serebryakov, Cong Xu, Can Li, Paolo Faraboschi, John Paul Strachan, Deming Chen, Kaushik Roy, Wen Mei Hwu, Dejan Milojicic

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

ReRAM-based accelerators have shown great potential for accelerating DNN inference because ReRAM crossbars can perform analog matrix-vector multiplication operations with low latency and energy consumption. However, these crossbars require the use of ADCs which constitute a significant fraction of the cost of MVM operations. The overhead of ADCs can be mitigated via partial sum quantization. However, prior quantization flows for DNN inference accelerators do not consider partial sum quantization which is not highly relevant to traditional digital architectures. To address this issue, we propose a mixed precision quantization scheme for ReRAM-based DNN inference accelerators where weight quantization, input quantization, and partial sum quantization are jointly applied for each DNN layer. We also propose an automated quantization flow powered by deep reinforcement learning to search for the best quantization configuration in the large design space. Our evaluation shows that the proposed mixed precision quantization scheme and quantization flow reduce inference latency and energy consumption by up to 3.89 and 4.84, respectively, while only losing 1.18% in DNN inference accuracy.

Original language	English (US)
Title of host publication	Proceedings of the 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	372-377
Number of pages	6
ISBN (Electronic)	9781450379991
DOIs	https://doi.org/10.1145/3394885.3431554
State	Published - Jan 18 2021
Event	26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021 - Virtual, Online, Japan Duration: Jan 18 2021 → Jan 21 2021

Publication series

Name	Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC

Conference

Conference	26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021
Country/Territory	Japan
City	Virtual, Online
Period	1/18/21 → 1/21/21

Keywords

DNN inference accelerators
Mixed precision quantization
ReRAM

ASJC Scopus subject areas

Electrical and Electronic Engineering
Computer Science Applications
Computer Graphics and Computer-Aided Design

Online availability

10.1145/3394885.3431554

Library availability

Discover UIUC Full Text

Cite this

Huang, S., Ankit, A., Silveira, P., Antunes, R., Chalamalasetti, S. R., Hajj, I. E., Kim, D. E., Aguiar, G., Bruel, P., Serebryakov, S., Xu, C., Li, C., Faraboschi, P., Strachan, J. P., Chen, D., Roy, K., Hwu, W. M., & Milojicic, D. (2021). Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators. In Proceedings of the 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021 (pp. 372-377). Article 3431554 (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/3394885.3431554

Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators. / Huang, Sitao; Ankit, Aayush; Silveira, Plinio et al.
Proceedings of the 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 372-377 3431554 (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Huang, S, Ankit, A, Silveira, P, Antunes, R, Chalamalasetti, SR, Hajj, IE, Kim, DE, Aguiar, G, Bruel, P, Serebryakov, S, Xu, C, Li, C, Faraboschi, P, Strachan, JP, Chen, D, Roy, K, Hwu, WM & Milojicic, D 2021, Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators. in Proceedings of the 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021., 3431554, Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC, Institute of Electrical and Electronics Engineers Inc., pp. 372-377, 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021, Virtual, Online, Japan, 1/18/21. https://doi.org/10.1145/3394885.3431554

Huang S, Ankit A, Silveira P, Antunes R, Chalamalasetti SR, Hajj IE et al. Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators. In Proceedings of the 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021. Institute of Electrical and Electronics Engineers Inc. 2021. p. 372-377. 3431554. (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC). doi: 10.1145/3394885.3431554

Huang, Sitao ; Ankit, Aayush ; Silveira, Plinio et al. / Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators. Proceedings of the 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021. Institute of Electrical and Electronics Engineers Inc., 2021. pp. 372-377 (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC).

@inproceedings{bfef6707d5804fa1a7ec1a72fc70217b,

title = "Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators",

abstract = "ReRAM-based accelerators have shown great potential for accelerating DNN inference because ReRAM crossbars can perform analog matrix-vector multiplication operations with low latency and energy consumption. However, these crossbars require the use of ADCs which constitute a significant fraction of the cost of MVM operations. The overhead of ADCs can be mitigated via partial sum quantization. However, prior quantization flows for DNN inference accelerators do not consider partial sum quantization which is not highly relevant to traditional digital architectures. To address this issue, we propose a mixed precision quantization scheme for ReRAM-based DNN inference accelerators where weight quantization, input quantization, and partial sum quantization are jointly applied for each DNN layer. We also propose an automated quantization flow powered by deep reinforcement learning to search for the best quantization configuration in the large design space. Our evaluation shows that the proposed mixed precision quantization scheme and quantization flow reduce inference latency and energy consumption by up to 3.89 and 4.84, respectively, while only losing 1.18% in DNN inference accuracy.",

keywords = "DNN inference accelerators, Mixed precision quantization, ReRAM",

author = "Sitao Huang and Aayush Ankit and Plinio Silveira and Rodrigo Antunes and Chalamalasetti, {Sai Rahul} and Hajj, {Izzat El} and Kim, {Dong Eun} and Glaucimar Aguiar and Pedro Bruel and Sergey Serebryakov and Cong Xu and Can Li and Paolo Faraboschi and Strachan, {John Paul} and Deming Chen and Kaushik Roy and Hwu, {Wen Mei} and Dejan Milojicic",

note = "Publisher Copyright: {\textcopyright} 2021 Association for Computing Machinery.; 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021 ; Conference date: 18-01-2021 Through 21-01-2021",

year = "2021",

month = jan,

day = "18",

doi = "10.1145/3394885.3431554",

language = "English (US)",

series = "Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "372--377",

booktitle = "Proceedings of the 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021",

address = "United States",

}

TY - GEN

T1 - Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators

AU - Huang, Sitao

AU - Ankit, Aayush

AU - Silveira, Plinio

AU - Antunes, Rodrigo

AU - Chalamalasetti, Sai Rahul

AU - Hajj, Izzat El

AU - Kim, Dong Eun

AU - Aguiar, Glaucimar

AU - Bruel, Pedro

AU - Serebryakov, Sergey

AU - Xu, Cong

AU - Li, Can

AU - Faraboschi, Paolo

AU - Strachan, John Paul

AU - Chen, Deming

AU - Roy, Kaushik

AU - Hwu, Wen Mei

AU - Milojicic, Dejan

PY - 2021/1/18

Y1 - 2021/1/18

N2 - ReRAM-based accelerators have shown great potential for accelerating DNN inference because ReRAM crossbars can perform analog matrix-vector multiplication operations with low latency and energy consumption. However, these crossbars require the use of ADCs which constitute a significant fraction of the cost of MVM operations. The overhead of ADCs can be mitigated via partial sum quantization. However, prior quantization flows for DNN inference accelerators do not consider partial sum quantization which is not highly relevant to traditional digital architectures. To address this issue, we propose a mixed precision quantization scheme for ReRAM-based DNN inference accelerators where weight quantization, input quantization, and partial sum quantization are jointly applied for each DNN layer. We also propose an automated quantization flow powered by deep reinforcement learning to search for the best quantization configuration in the large design space. Our evaluation shows that the proposed mixed precision quantization scheme and quantization flow reduce inference latency and energy consumption by up to 3.89 and 4.84, respectively, while only losing 1.18% in DNN inference accuracy.

AB - ReRAM-based accelerators have shown great potential for accelerating DNN inference because ReRAM crossbars can perform analog matrix-vector multiplication operations with low latency and energy consumption. However, these crossbars require the use of ADCs which constitute a significant fraction of the cost of MVM operations. The overhead of ADCs can be mitigated via partial sum quantization. However, prior quantization flows for DNN inference accelerators do not consider partial sum quantization which is not highly relevant to traditional digital architectures. To address this issue, we propose a mixed precision quantization scheme for ReRAM-based DNN inference accelerators where weight quantization, input quantization, and partial sum quantization are jointly applied for each DNN layer. We also propose an automated quantization flow powered by deep reinforcement learning to search for the best quantization configuration in the large design space. Our evaluation shows that the proposed mixed precision quantization scheme and quantization flow reduce inference latency and energy consumption by up to 3.89 and 4.84, respectively, while only losing 1.18% in DNN inference accuracy.

KW - DNN inference accelerators

KW - Mixed precision quantization

KW - ReRAM

UR - http://www.scopus.com/inward/record.url?scp=85100574970&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85100574970&partnerID=8YFLogxK

U2 - 10.1145/3394885.3431554

DO - 10.1145/3394885.3431554

M3 - Conference contribution

AN - SCOPUS:85100574970

T3 - Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC

SP - 372

EP - 377

BT - Proceedings of the 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 26th Asia and South Pacific Design Automation Conference, ASP-DAC 2021

Y2 - 18 January 2021 through 21 January 2021

ER -

Mixed Precision Quantization for ReRAM-based DNN Inference Accelerators

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Online availability

Library availability

Related links

Fingerprint

Cite this