Stardust: A deep learning serving system in IoT: Demo abstract

Shuochao Yao, Tianshi Wang, Jinyang Li, Tarek Abdelzaher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The deep neural network becomes an increasingly crucial component in recent intelligent applications. The excessive resource consumptions of state-of-the-art neural networks, however, remains a huge impediment towards their widespread deployment in the Internet of Things (IoT). In this paper, we propose an IoT-oriented deep learning serving system, Stardust, that accelerates the neural network inference to improve the quality of IoT services. Stardust integrates several joint contributions from both the system and AI perspectives, including system performance predictor, model compression, and compressive offloading. On one hand, the performance predictor profiles and predicts the runtime characteristics of neural network operations on a particular device with the targeted runtime environment, which enables a hardware and software oriented performance optimization during model compression and offloading. On the other hand, the model compression minimizes the computation time of neural networks on different devices, and the compressive offloading diminishes the network data transferring time during the mobile-edge offloading. Moreover, all these optimizations can be done with almost no compromise on inference accuracy. The integration of these modules, therefore, collaboratively reduce the end-to-end latency of serving deep learning services that reside across embedded/mobile devices and edge servers. We deploy illustrative applications on Stardust, performing human perception tasks with on-device camera microphone and motion sensors to demonstrate the capability of Stardust serving system.

Original languageEnglish (US)
Title of host publicationSenSys 2019 - Proceedings of the 17th Conference on Embedded Networked Sensor Systems
EditorsMi Zhang
PublisherAssociation for Computing Machinery, Inc
Pages402-403
Number of pages2
ISBN (Electronic)9781450369503
DOIs
StatePublished - Nov 10 2019
Event17th ACM Conference on Embedded Networked Sensor Systems, SenSys 2019 - New York, United States
Duration: Nov 10 2019Nov 13 2019

Publication series

NameSenSys 2019 - Proceedings of the 17th Conference on Embedded Networked Sensor Systems

Conference

Conference17th ACM Conference on Embedded Networked Sensor Systems, SenSys 2019
CountryUnited States
CityNew York
Period11/10/1911/13/19

Keywords

  • Deep learning
  • IoT
  • Model compression
  • Offloading

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Stardust: A deep learning serving system in IoT: Demo abstract'. Together they form a unique fingerprint.

  • Cite this

    Yao, S., Wang, T., Li, J., & Abdelzaher, T. (2019). Stardust: A deep learning serving system in IoT: Demo abstract. In M. Zhang (Ed.), SenSys 2019 - Proceedings of the 17th Conference on Embedded Networked Sensor Systems (pp. 402-403). (SenSys 2019 - Proceedings of the 17th Conference on Embedded Networked Sensor Systems). Association for Computing Machinery, Inc. https://doi.org/10.1145/3356250.3361962