FastDeepIoT: Towards understanding and optimizing neural network execution time on mobile and embedded devices

Shuochao Yao, Yiran Zhao, Huajie Shao, Sheng Zhong Liu, Dongxin Liu, Lu Su, Tarek Abdelzaher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve performance. We show that changing neural network size does not proportionally affect performance attributes of interest, such as execution time. Rather, extreme run-time nonlinearities exist over the network configuration space. Hence, we propose a novel framework, called FastDeepIoT, that uncovers the non-linear relation between neural network structure and execution time, then exploits that understanding to find network configurations that significantly improve the trade-off between execution time and accuracy on mobile and embedded devices. FastDeepIoT makes two key contributions. First, FastDeepIoT automatically learns an accurate and highly interpretable execution time model for deep neural networks on the target device. This is done without prior knowledge of either the hardware specifications or the detailed implementation of the used deep learning library. Second, FastDeepIoT informs a

Original languageEnglish (US)
Title of host publicationSenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems
PublisherAssociation for Computing Machinery, Inc
Pages278-291
Number of pages14
ISBN (Electronic)9781450359528
DOIs
StatePublished - Nov 4 2018
Event16th ACM Conference on Embedded Networked Sensor Systems, SENSYS 2018 - Shenzhen, China
Duration: Nov 4 2018Nov 7 2018

Publication series

NameSenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems

Conference

Conference16th ACM Conference on Embedded Networked Sensor Systems, SENSYS 2018
CountryChina
CityShenzhen
Period11/4/1811/7/18

    Fingerprint

Keywords

  • Deep Learning
  • Execution Time
  • Internet of Things
  • Mobile Computing
  • Model Compression

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Computer Networks and Communications

Cite this

Yao, S., Zhao, Y., Shao, H., Liu, S. Z., Liu, D., Su, L., & Abdelzaher, T. (2018). FastDeepIoT: Towards understanding and optimizing neural network execution time on mobile and embedded devices. In SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems (pp. 278-291). (SenSys 2018 - Proceedings of the 16th Conference on Embedded Networked Sensor Systems). Association for Computing Machinery, Inc. https://doi.org/10.1145/3274783.3274840