Energy-reliability limits in nanoscale neural networks

Avhishek Chatterjee, Lav R Varshney

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

New device technologies such as spintronics, carbon nanotubes, and nanoscale CMOS incur random transient failures, where the failure probability is governed by the energy consumption through energy-failure functions. At the same time, there is growing use of deep neural networks for many inference applications, and specialized hardware is being developed with these nanotechnologies as physical substrates. It is important to understand the basic energy-reliability limits. Using Pippenger's mutual information propagation technique (extended to directed acyclic graphs), together with optimization, we obtain a lower bound on energy consumption in multilayer binary neural networks for a given reliability. We also obtain a simple energy allocation rule for neurons in the different layers of the neural network. The mathematical results also provide insight into mammalian neuroenergetics of brain regions involved in sensory processing.

Original languageEnglish (US)
Title of host publication2017 51st Annual Conference on Information Sciences and Systems, CISS 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509047802
DOIs
StatePublished - May 10 2017
Event51st Annual Conference on Information Sciences and Systems, CISS 2017 - Baltimore, United States
Duration: Mar 22 2017Mar 24 2017

Publication series

Name2017 51st Annual Conference on Information Sciences and Systems, CISS 2017

Other

Other51st Annual Conference on Information Sciences and Systems, CISS 2017
CountryUnited States
CityBaltimore
Period3/22/173/24/17

Fingerprint

Energy utilization
Neural networks
Magnetoelectronics
Nanotechnology
Neurons
Carbon nanotubes
Brain
Multilayers
Hardware
Substrates
Processing
Energy
Energy consumption
Deep neural networks
Lower bounds
Directed acyclic graph
Substrate
Neuron
Inference
Allocation rules

Keywords

  • Energy-failure function
  • Neural network
  • Reliability

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems and Management
  • Computer Networks and Communications
  • Information Systems

Cite this

Chatterjee, A., & Varshney, L. R. (2017). Energy-reliability limits in nanoscale neural networks. In 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017 [7926139] (2017 51st Annual Conference on Information Sciences and Systems, CISS 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CISS.2017.7926139

Energy-reliability limits in nanoscale neural networks. / Chatterjee, Avhishek; Varshney, Lav R.

2017 51st Annual Conference on Information Sciences and Systems, CISS 2017. Institute of Electrical and Electronics Engineers Inc., 2017. 7926139 (2017 51st Annual Conference on Information Sciences and Systems, CISS 2017).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Chatterjee, A & Varshney, LR 2017, Energy-reliability limits in nanoscale neural networks. in 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017., 7926139, 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017, Institute of Electrical and Electronics Engineers Inc., 51st Annual Conference on Information Sciences and Systems, CISS 2017, Baltimore, United States, 3/22/17. https://doi.org/10.1109/CISS.2017.7926139
Chatterjee A, Varshney LR. Energy-reliability limits in nanoscale neural networks. In 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017. Institute of Electrical and Electronics Engineers Inc. 2017. 7926139. (2017 51st Annual Conference on Information Sciences and Systems, CISS 2017). https://doi.org/10.1109/CISS.2017.7926139
Chatterjee, Avhishek ; Varshney, Lav R. / Energy-reliability limits in nanoscale neural networks. 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017. Institute of Electrical and Electronics Engineers Inc., 2017. (2017 51st Annual Conference on Information Sciences and Systems, CISS 2017).
@inproceedings{883988cdf50c48f2bb277e8a839801af,
title = "Energy-reliability limits in nanoscale neural networks",
abstract = "New device technologies such as spintronics, carbon nanotubes, and nanoscale CMOS incur random transient failures, where the failure probability is governed by the energy consumption through energy-failure functions. At the same time, there is growing use of deep neural networks for many inference applications, and specialized hardware is being developed with these nanotechnologies as physical substrates. It is important to understand the basic energy-reliability limits. Using Pippenger's mutual information propagation technique (extended to directed acyclic graphs), together with optimization, we obtain a lower bound on energy consumption in multilayer binary neural networks for a given reliability. We also obtain a simple energy allocation rule for neurons in the different layers of the neural network. The mathematical results also provide insight into mammalian neuroenergetics of brain regions involved in sensory processing.",
keywords = "Energy-failure function, Neural network, Reliability",
author = "Avhishek Chatterjee and Varshney, {Lav R}",
year = "2017",
month = "5",
day = "10",
doi = "10.1109/CISS.2017.7926139",
language = "English (US)",
series = "2017 51st Annual Conference on Information Sciences and Systems, CISS 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2017 51st Annual Conference on Information Sciences and Systems, CISS 2017",
address = "United States",

}

TY - GEN

T1 - Energy-reliability limits in nanoscale neural networks

AU - Chatterjee, Avhishek

AU - Varshney, Lav R

PY - 2017/5/10

Y1 - 2017/5/10

N2 - New device technologies such as spintronics, carbon nanotubes, and nanoscale CMOS incur random transient failures, where the failure probability is governed by the energy consumption through energy-failure functions. At the same time, there is growing use of deep neural networks for many inference applications, and specialized hardware is being developed with these nanotechnologies as physical substrates. It is important to understand the basic energy-reliability limits. Using Pippenger's mutual information propagation technique (extended to directed acyclic graphs), together with optimization, we obtain a lower bound on energy consumption in multilayer binary neural networks for a given reliability. We also obtain a simple energy allocation rule for neurons in the different layers of the neural network. The mathematical results also provide insight into mammalian neuroenergetics of brain regions involved in sensory processing.

AB - New device technologies such as spintronics, carbon nanotubes, and nanoscale CMOS incur random transient failures, where the failure probability is governed by the energy consumption through energy-failure functions. At the same time, there is growing use of deep neural networks for many inference applications, and specialized hardware is being developed with these nanotechnologies as physical substrates. It is important to understand the basic energy-reliability limits. Using Pippenger's mutual information propagation technique (extended to directed acyclic graphs), together with optimization, we obtain a lower bound on energy consumption in multilayer binary neural networks for a given reliability. We also obtain a simple energy allocation rule for neurons in the different layers of the neural network. The mathematical results also provide insight into mammalian neuroenergetics of brain regions involved in sensory processing.

KW - Energy-failure function

KW - Neural network

KW - Reliability

UR - http://www.scopus.com/inward/record.url?scp=85020187299&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020187299&partnerID=8YFLogxK

U2 - 10.1109/CISS.2017.7926139

DO - 10.1109/CISS.2017.7926139

M3 - Conference contribution

AN - SCOPUS:85020187299

T3 - 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017

BT - 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -