DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework

Shuochao Yao, Yiran Zhao, Aston Zhang, Lu Su, Tarek Abdelzaher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recent advances in deep learning motivate the use of deep neutral networks in sensing applications, but their excessive resource needs on constrained embedded devices remain an important impediment. A recently explored solution space lies in compressing (approximating or simplifying) deep neural networks in some manner before use on the device. We propose a new compression solution, called DeepIoT, that makes two key contributions in that space. First, unlike current solutions geared for compressing specific types of neural networks, DeepIoT presents a unified approach that compresses all commonly used deep learning structures for sensing applications, including fully-connected, convolutional, and recurrent neural networks, as well as their combinations. Second, unlike solutions that either sparsify weight matrices or assume linear structure within weight matrices, DeepIoT compresses neural network structures into smaller dense matrices by finding the minimum number of non-redundant hidden elements, such as filters and dimensions required by each layer, while keeping the performance of sensing applications the same. Importantly, it does so using an approach that obtains a global view of parameter redundancies, which is shown to produce superior compression. The compressed model generated by DeepIoT can directly use existing deep learning libraries that run on embedded and mobile systems without further modifications. We conduct experiments with five different sensing-related tasks on Intel Edison devices. DeepIoT outperforms all compared baseline algorithms with respect to execution time and energy consumption by a significant margin. It reduces the size of deep neural networks by 90% to 98.9%. It is thus able to shorten execution time by 71.4% to 94.5%, and decrease energy consumption by 72.2% to 95.7%. These improvements are achieved without loss of accuracy. The results underscore the potential of DeepIoT for advancing the exploitation of deep neural networks on resource-constrained embedded devices.

Original languageEnglish (US)
Title of host publicationSenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems
EditorsRasit Eskicioglu
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450354592
DOIs
StatePublished - Nov 6 2017
Event15th ACM Conference on Embedded Networked Sensor Systems, SenSys 2017 - Delft, Netherlands
Duration: Nov 6 2017Nov 8 2017

Publication series

NameSenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems
Volume2017-January

Other

Other15th ACM Conference on Embedded Networked Sensor Systems, SenSys 2017
CountryNetherlands
CityDelft
Period11/6/1711/8/17

Fingerprint

Compressors
Energy utilization
Neural networks
Recurrent neural networks
Redundancy
Compaction
Deep neural networks
Deep learning
Experiments

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Computer Networks and Communications

Cite this

Yao, S., Zhao, Y., Zhang, A., Su, L., & Abdelzaher, T. (2017). DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework. In R. Eskicioglu (Ed.), SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems (SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems; Vol. 2017-January). Association for Computing Machinery, Inc. https://doi.org/10.1145/3131672.3131675

DeepIoT : Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework. / Yao, Shuochao; Zhao, Yiran; Zhang, Aston; Su, Lu; Abdelzaher, Tarek.

SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems. ed. / Rasit Eskicioglu. Association for Computing Machinery, Inc, 2017. (SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems; Vol. 2017-January).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yao, S, Zhao, Y, Zhang, A, Su, L & Abdelzaher, T 2017, DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework. in R Eskicioglu (ed.), SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems. SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems, vol. 2017-January, Association for Computing Machinery, Inc, 15th ACM Conference on Embedded Networked Sensor Systems, SenSys 2017, Delft, Netherlands, 11/6/17. https://doi.org/10.1145/3131672.3131675
Yao S, Zhao Y, Zhang A, Su L, Abdelzaher T. DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework. In Eskicioglu R, editor, SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems. Association for Computing Machinery, Inc. 2017. (SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems). https://doi.org/10.1145/3131672.3131675
Yao, Shuochao ; Zhao, Yiran ; Zhang, Aston ; Su, Lu ; Abdelzaher, Tarek. / DeepIoT : Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework. SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems. editor / Rasit Eskicioglu. Association for Computing Machinery, Inc, 2017. (SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems).
@inproceedings{1b5067c93f334b6985cb0e68fc3e81fb,
title = "DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework",
abstract = "Recent advances in deep learning motivate the use of deep neutral networks in sensing applications, but their excessive resource needs on constrained embedded devices remain an important impediment. A recently explored solution space lies in compressing (approximating or simplifying) deep neural networks in some manner before use on the device. We propose a new compression solution, called DeepIoT, that makes two key contributions in that space. First, unlike current solutions geared for compressing specific types of neural networks, DeepIoT presents a unified approach that compresses all commonly used deep learning structures for sensing applications, including fully-connected, convolutional, and recurrent neural networks, as well as their combinations. Second, unlike solutions that either sparsify weight matrices or assume linear structure within weight matrices, DeepIoT compresses neural network structures into smaller dense matrices by finding the minimum number of non-redundant hidden elements, such as filters and dimensions required by each layer, while keeping the performance of sensing applications the same. Importantly, it does so using an approach that obtains a global view of parameter redundancies, which is shown to produce superior compression. The compressed model generated by DeepIoT can directly use existing deep learning libraries that run on embedded and mobile systems without further modifications. We conduct experiments with five different sensing-related tasks on Intel Edison devices. DeepIoT outperforms all compared baseline algorithms with respect to execution time and energy consumption by a significant margin. It reduces the size of deep neural networks by 90{\%} to 98.9{\%}. It is thus able to shorten execution time by 71.4{\%} to 94.5{\%}, and decrease energy consumption by 72.2{\%} to 95.7{\%}. These improvements are achieved without loss of accuracy. The results underscore the potential of DeepIoT for advancing the exploitation of deep neural networks on resource-constrained embedded devices.",
author = "Shuochao Yao and Yiran Zhao and Aston Zhang and Lu Su and Tarek Abdelzaher",
year = "2017",
month = "11",
day = "6",
doi = "10.1145/3131672.3131675",
language = "English (US)",
series = "SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems",
publisher = "Association for Computing Machinery, Inc",
editor = "Rasit Eskicioglu",
booktitle = "SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems",

}

TY - GEN

T1 - DeepIoT

T2 - Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework

AU - Yao, Shuochao

AU - Zhao, Yiran

AU - Zhang, Aston

AU - Su, Lu

AU - Abdelzaher, Tarek

PY - 2017/11/6

Y1 - 2017/11/6

N2 - Recent advances in deep learning motivate the use of deep neutral networks in sensing applications, but their excessive resource needs on constrained embedded devices remain an important impediment. A recently explored solution space lies in compressing (approximating or simplifying) deep neural networks in some manner before use on the device. We propose a new compression solution, called DeepIoT, that makes two key contributions in that space. First, unlike current solutions geared for compressing specific types of neural networks, DeepIoT presents a unified approach that compresses all commonly used deep learning structures for sensing applications, including fully-connected, convolutional, and recurrent neural networks, as well as their combinations. Second, unlike solutions that either sparsify weight matrices or assume linear structure within weight matrices, DeepIoT compresses neural network structures into smaller dense matrices by finding the minimum number of non-redundant hidden elements, such as filters and dimensions required by each layer, while keeping the performance of sensing applications the same. Importantly, it does so using an approach that obtains a global view of parameter redundancies, which is shown to produce superior compression. The compressed model generated by DeepIoT can directly use existing deep learning libraries that run on embedded and mobile systems without further modifications. We conduct experiments with five different sensing-related tasks on Intel Edison devices. DeepIoT outperforms all compared baseline algorithms with respect to execution time and energy consumption by a significant margin. It reduces the size of deep neural networks by 90% to 98.9%. It is thus able to shorten execution time by 71.4% to 94.5%, and decrease energy consumption by 72.2% to 95.7%. These improvements are achieved without loss of accuracy. The results underscore the potential of DeepIoT for advancing the exploitation of deep neural networks on resource-constrained embedded devices.

AB - Recent advances in deep learning motivate the use of deep neutral networks in sensing applications, but their excessive resource needs on constrained embedded devices remain an important impediment. A recently explored solution space lies in compressing (approximating or simplifying) deep neural networks in some manner before use on the device. We propose a new compression solution, called DeepIoT, that makes two key contributions in that space. First, unlike current solutions geared for compressing specific types of neural networks, DeepIoT presents a unified approach that compresses all commonly used deep learning structures for sensing applications, including fully-connected, convolutional, and recurrent neural networks, as well as their combinations. Second, unlike solutions that either sparsify weight matrices or assume linear structure within weight matrices, DeepIoT compresses neural network structures into smaller dense matrices by finding the minimum number of non-redundant hidden elements, such as filters and dimensions required by each layer, while keeping the performance of sensing applications the same. Importantly, it does so using an approach that obtains a global view of parameter redundancies, which is shown to produce superior compression. The compressed model generated by DeepIoT can directly use existing deep learning libraries that run on embedded and mobile systems without further modifications. We conduct experiments with five different sensing-related tasks on Intel Edison devices. DeepIoT outperforms all compared baseline algorithms with respect to execution time and energy consumption by a significant margin. It reduces the size of deep neural networks by 90% to 98.9%. It is thus able to shorten execution time by 71.4% to 94.5%, and decrease energy consumption by 72.2% to 95.7%. These improvements are achieved without loss of accuracy. The results underscore the potential of DeepIoT for advancing the exploitation of deep neural networks on resource-constrained embedded devices.

UR - http://www.scopus.com/inward/record.url?scp=85050189919&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050189919&partnerID=8YFLogxK

U2 - 10.1145/3131672.3131675

DO - 10.1145/3131672.3131675

M3 - Conference contribution

AN - SCOPUS:85050189919

T3 - SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems

BT - SenSys 2017 - Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems

A2 - Eskicioglu, Rasit

PB - Association for Computing Machinery, Inc

ER -