FastDeepIoT: Towards understanding and optimizing neural network execution time on mobile and embedded devices

Shuochao Yao, Yiran Zhao, Huajie Shao, Sheng Zhong Liu, Dongxin Liu, Lu Su, Tarek Abdelzaher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve performance. We show that changing neural network size does not proportionally affect performance attributes of interest, such as execution time. Rather, extreme run-time nonlinearities exist over the network configuration space. Hence, we propose a novel framework, called FastDeepIoT, that uncovers the non-linear relation between neural network structure and execution time, then exploits that understanding to find network configurations that significantly improve the trade-off between execution time and accuracy on mobile and embedded devices. FastDeepIoT makes two key contributions. First, FastDeepIoT automatically learns an accurate and highly interpretable execution time model for deep neural networks on the target device. This is done without prior knowledge of either the hardware specifications or the detailed implementation of the used deep learning library. Second, FastDeepIoT informs a

Original languageEnglish (US)
Title of host publicationSenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems
PublisherAssociation for Computing Machinery, Inc
Pages278-291
Number of pages14
ISBN (Electronic)9781450359528
DOIs
StatePublished - Nov 4 2018
Event16th ACM Conference on Embedded Networked Sensor Systems, SENSYS 2018 - Shenzhen, China
Duration: Nov 4 2018Nov 7 2018

Publication series

NameSenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems

Conference

Conference16th ACM Conference on Embedded Networked Sensor Systems, SENSYS 2018
CountryChina
CityShenzhen
Period11/4/1811/7/18

Fingerprint

Neural networks
Specifications
Hardware
Deep neural networks
Deep learning

Keywords

  • Deep Learning
  • Execution Time
  • Internet of Things
  • Mobile Computing
  • Model Compression

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Computer Networks and Communications

Cite this

Yao, S., Zhao, Y., Shao, H., Liu, S. Z., Liu, D., Su, L., & Abdelzaher, T. (2018). FastDeepIoT: Towards understanding and optimizing neural network execution time on mobile and embedded devices. In SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems (pp. 278-291). (SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems). Association for Computing Machinery, Inc. https://doi.org/10.1145/3274783.3274840

FastDeepIoT : Towards understanding and optimizing neural network execution time on mobile and embedded devices. / Yao, Shuochao; Zhao, Yiran; Shao, Huajie; Liu, Sheng Zhong; Liu, Dongxin; Su, Lu; Abdelzaher, Tarek.

SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems. Association for Computing Machinery, Inc, 2018. p. 278-291 (SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yao, S, Zhao, Y, Shao, H, Liu, SZ, Liu, D, Su, L & Abdelzaher, T 2018, FastDeepIoT: Towards understanding and optimizing neural network execution time on mobile and embedded devices. in SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems. SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems, Association for Computing Machinery, Inc, pp. 278-291, 16th ACM Conference on Embedded Networked Sensor Systems, SENSYS 2018, Shenzhen, China, 11/4/18. https://doi.org/10.1145/3274783.3274840
Yao S, Zhao Y, Shao H, Liu SZ, Liu D, Su L et al. FastDeepIoT: Towards understanding and optimizing neural network execution time on mobile and embedded devices. In SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems. Association for Computing Machinery, Inc. 2018. p. 278-291. (SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems). https://doi.org/10.1145/3274783.3274840
Yao, Shuochao ; Zhao, Yiran ; Shao, Huajie ; Liu, Sheng Zhong ; Liu, Dongxin ; Su, Lu ; Abdelzaher, Tarek. / FastDeepIoT : Towards understanding and optimizing neural network execution time on mobile and embedded devices. SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems. Association for Computing Machinery, Inc, 2018. pp. 278-291 (SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems).
@inproceedings{ef7b331143fb4943bb94e8b0cd02fe23,
title = "FastDeepIoT: Towards understanding and optimizing neural network execution time on mobile and embedded devices",
abstract = "Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve performance. We show that changing neural network size does not proportionally affect performance attributes of interest, such as execution time. Rather, extreme run-time nonlinearities exist over the network configuration space. Hence, we propose a novel framework, called FastDeepIoT, that uncovers the non-linear relation between neural network structure and execution time, then exploits that understanding to find network configurations that significantly improve the trade-off between execution time and accuracy on mobile and embedded devices. FastDeepIoT makes two key contributions. First, FastDeepIoT automatically learns an accurate and highly interpretable execution time model for deep neural networks on the target device. This is done without prior knowledge of either the hardware specifications or the detailed implementation of the used deep learning library. Second, FastDeepIoT informs a",
keywords = "Deep Learning, Execution Time, Internet of Things, Mobile Computing, Model Compression",
author = "Shuochao Yao and Yiran Zhao and Huajie Shao and Liu, {Sheng Zhong} and Dongxin Liu and Lu Su and Tarek Abdelzaher",
year = "2018",
month = "11",
day = "4",
doi = "10.1145/3274783.3274840",
language = "English (US)",
series = "SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems",
publisher = "Association for Computing Machinery, Inc",
pages = "278--291",
booktitle = "SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems",

}

TY - GEN

T1 - FastDeepIoT

T2 - Towards understanding and optimizing neural network execution time on mobile and embedded devices

AU - Yao, Shuochao

AU - Zhao, Yiran

AU - Shao, Huajie

AU - Liu, Sheng Zhong

AU - Liu, Dongxin

AU - Su, Lu

AU - Abdelzaher, Tarek

PY - 2018/11/4

Y1 - 2018/11/4

N2 - Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve performance. We show that changing neural network size does not proportionally affect performance attributes of interest, such as execution time. Rather, extreme run-time nonlinearities exist over the network configuration space. Hence, we propose a novel framework, called FastDeepIoT, that uncovers the non-linear relation between neural network structure and execution time, then exploits that understanding to find network configurations that significantly improve the trade-off between execution time and accuracy on mobile and embedded devices. FastDeepIoT makes two key contributions. First, FastDeepIoT automatically learns an accurate and highly interpretable execution time model for deep neural networks on the target device. This is done without prior knowledge of either the hardware specifications or the detailed implementation of the used deep learning library. Second, FastDeepIoT informs a

AB - Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve performance. We show that changing neural network size does not proportionally affect performance attributes of interest, such as execution time. Rather, extreme run-time nonlinearities exist over the network configuration space. Hence, we propose a novel framework, called FastDeepIoT, that uncovers the non-linear relation between neural network structure and execution time, then exploits that understanding to find network configurations that significantly improve the trade-off between execution time and accuracy on mobile and embedded devices. FastDeepIoT makes two key contributions. First, FastDeepIoT automatically learns an accurate and highly interpretable execution time model for deep neural networks on the target device. This is done without prior knowledge of either the hardware specifications or the detailed implementation of the used deep learning library. Second, FastDeepIoT informs a

KW - Deep Learning

KW - Execution Time

KW - Internet of Things

KW - Mobile Computing

KW - Model Compression

UR - http://www.scopus.com/inward/record.url?scp=85061719356&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061719356&partnerID=8YFLogxK

U2 - 10.1145/3274783.3274840

DO - 10.1145/3274783.3274840

M3 - Conference contribution

AN - SCOPUS:85061719356

T3 - SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems

SP - 278

EP - 291

BT - SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems

PB - Association for Computing Machinery, Inc

ER -