SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications

Shuochao Yao, Yiran Zhao, Huajie Shao, Dongxin Liu, Shengzhong Liu, Yifan Hao, Ailing Piao, Shaohan Hu, Su Lu, Tarek Abdelzaher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep neural networks are becoming increasingly popular in Internet of Things (IoT) applications. Their capabilities of fusing multiple sensor inputs and extracting temporal relationships can enhance intelligence in a wide range of applications. However, one key problem is the missing of adaptation to heterogeneous on-device sensors. These low-end sensors on IoT devices possess different accuracies, granularities, and amounts of information, whose sensing qualities are heterogeneous and vary over time. The existing deep learning frameworks for IoT applications usually treat every sensor input equally over time or increase model capacity in an ad-hoc manner, lacking the ability to identify and exploit the sensor heterogeneities. In this work, we propose SADeepSense, a deep learning framework that can automatically balance the contributions of multiple sensor inputs over time by exploiting their sensing qualities. SADeepSense makes two key contributions. First, SADeepSense employs the self-attention mechanism to learn the correlations among different sensors over time with no additional supervision. The correlations are then applied to infer the sensing qualities and to reassign model concentrations in multiple sensors over time. Second, instead of directly learning the sensing qualities and contributions, SADeepSense generates the residual concentrations that are deviated from the equal contributions, which helps to stabilize the training process. We demonstrate the effectiveness of SADeepSense with two representative IoT sensing tasks: heterogeneous human activity recognition with motion sensors and gesture recognition with the wireless signal. SADeepSense consistently outperforms the state-of-the-art methods by a clear margin. In addition, we show that SADeepSense only imposes little additional resource-consumption burden on embedded devices compared to the corresponding state-of-the-art framework.

Original languageEnglish (US)
Title of host publicationINFOCOM 2019 - IEEE Conference on Computer Communications
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1243-1251
Number of pages9
ISBN (Electronic)9781728105154
DOIs
StatePublished - Apr 2019
Event2019 IEEE Conference on Computer Communications, INFOCOM 2019 - Paris, France
Duration: Apr 29 2019May 2 2019

Publication series

NameProceedings - IEEE INFOCOM
Volume2019-April
ISSN (Print)0743-166X

Conference

Conference2019 IEEE Conference on Computer Communications, INFOCOM 2019
CountryFrance
CityParis
Period4/29/195/2/19

Fingerprint

Sensors
Deep learning
Internet of things
Gesture recognition

ASJC Scopus subject areas

  • Computer Science(all)
  • Electrical and Electronic Engineering

Cite this

Yao, S., Zhao, Y., Shao, H., Liu, D., Liu, S., Hao, Y., ... Abdelzaher, T. (2019). SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications. In INFOCOM 2019 - IEEE Conference on Computer Communications (pp. 1243-1251). [8737500] (Proceedings - IEEE INFOCOM; Vol. 2019-April). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/INFOCOM.2019.8737500

SADeepSense : Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications. / Yao, Shuochao; Zhao, Yiran; Shao, Huajie; Liu, Dongxin; Liu, Shengzhong; Hao, Yifan; Piao, Ailing; Hu, Shaohan; Lu, Su; Abdelzaher, Tarek.

INFOCOM 2019 - IEEE Conference on Computer Communications. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1243-1251 8737500 (Proceedings - IEEE INFOCOM; Vol. 2019-April).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yao, S, Zhao, Y, Shao, H, Liu, D, Liu, S, Hao, Y, Piao, A, Hu, S, Lu, S & Abdelzaher, T 2019, SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications. in INFOCOM 2019 - IEEE Conference on Computer Communications., 8737500, Proceedings - IEEE INFOCOM, vol. 2019-April, Institute of Electrical and Electronics Engineers Inc., pp. 1243-1251, 2019 IEEE Conference on Computer Communications, INFOCOM 2019, Paris, France, 4/29/19. https://doi.org/10.1109/INFOCOM.2019.8737500
Yao S, Zhao Y, Shao H, Liu D, Liu S, Hao Y et al. SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications. In INFOCOM 2019 - IEEE Conference on Computer Communications. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1243-1251. 8737500. (Proceedings - IEEE INFOCOM). https://doi.org/10.1109/INFOCOM.2019.8737500
Yao, Shuochao ; Zhao, Yiran ; Shao, Huajie ; Liu, Dongxin ; Liu, Shengzhong ; Hao, Yifan ; Piao, Ailing ; Hu, Shaohan ; Lu, Su ; Abdelzaher, Tarek. / SADeepSense : Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications. INFOCOM 2019 - IEEE Conference on Computer Communications. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1243-1251 (Proceedings - IEEE INFOCOM).
@inproceedings{f623e4950c544aacbd85bf8bf86f8ccd,
title = "SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications",
abstract = "Deep neural networks are becoming increasingly popular in Internet of Things (IoT) applications. Their capabilities of fusing multiple sensor inputs and extracting temporal relationships can enhance intelligence in a wide range of applications. However, one key problem is the missing of adaptation to heterogeneous on-device sensors. These low-end sensors on IoT devices possess different accuracies, granularities, and amounts of information, whose sensing qualities are heterogeneous and vary over time. The existing deep learning frameworks for IoT applications usually treat every sensor input equally over time or increase model capacity in an ad-hoc manner, lacking the ability to identify and exploit the sensor heterogeneities. In this work, we propose SADeepSense, a deep learning framework that can automatically balance the contributions of multiple sensor inputs over time by exploiting their sensing qualities. SADeepSense makes two key contributions. First, SADeepSense employs the self-attention mechanism to learn the correlations among different sensors over time with no additional supervision. The correlations are then applied to infer the sensing qualities and to reassign model concentrations in multiple sensors over time. Second, instead of directly learning the sensing qualities and contributions, SADeepSense generates the residual concentrations that are deviated from the equal contributions, which helps to stabilize the training process. We demonstrate the effectiveness of SADeepSense with two representative IoT sensing tasks: heterogeneous human activity recognition with motion sensors and gesture recognition with the wireless signal. SADeepSense consistently outperforms the state-of-the-art methods by a clear margin. In addition, we show that SADeepSense only imposes little additional resource-consumption burden on embedded devices compared to the corresponding state-of-the-art framework.",
author = "Shuochao Yao and Yiran Zhao and Huajie Shao and Dongxin Liu and Shengzhong Liu and Yifan Hao and Ailing Piao and Shaohan Hu and Su Lu and Tarek Abdelzaher",
year = "2019",
month = "4",
doi = "10.1109/INFOCOM.2019.8737500",
language = "English (US)",
series = "Proceedings - IEEE INFOCOM",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1243--1251",
booktitle = "INFOCOM 2019 - IEEE Conference on Computer Communications",
address = "United States",

}

TY - GEN

T1 - SADeepSense

T2 - Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications

AU - Yao, Shuochao

AU - Zhao, Yiran

AU - Shao, Huajie

AU - Liu, Dongxin

AU - Liu, Shengzhong

AU - Hao, Yifan

AU - Piao, Ailing

AU - Hu, Shaohan

AU - Lu, Su

AU - Abdelzaher, Tarek

PY - 2019/4

Y1 - 2019/4

N2 - Deep neural networks are becoming increasingly popular in Internet of Things (IoT) applications. Their capabilities of fusing multiple sensor inputs and extracting temporal relationships can enhance intelligence in a wide range of applications. However, one key problem is the missing of adaptation to heterogeneous on-device sensors. These low-end sensors on IoT devices possess different accuracies, granularities, and amounts of information, whose sensing qualities are heterogeneous and vary over time. The existing deep learning frameworks for IoT applications usually treat every sensor input equally over time or increase model capacity in an ad-hoc manner, lacking the ability to identify and exploit the sensor heterogeneities. In this work, we propose SADeepSense, a deep learning framework that can automatically balance the contributions of multiple sensor inputs over time by exploiting their sensing qualities. SADeepSense makes two key contributions. First, SADeepSense employs the self-attention mechanism to learn the correlations among different sensors over time with no additional supervision. The correlations are then applied to infer the sensing qualities and to reassign model concentrations in multiple sensors over time. Second, instead of directly learning the sensing qualities and contributions, SADeepSense generates the residual concentrations that are deviated from the equal contributions, which helps to stabilize the training process. We demonstrate the effectiveness of SADeepSense with two representative IoT sensing tasks: heterogeneous human activity recognition with motion sensors and gesture recognition with the wireless signal. SADeepSense consistently outperforms the state-of-the-art methods by a clear margin. In addition, we show that SADeepSense only imposes little additional resource-consumption burden on embedded devices compared to the corresponding state-of-the-art framework.

AB - Deep neural networks are becoming increasingly popular in Internet of Things (IoT) applications. Their capabilities of fusing multiple sensor inputs and extracting temporal relationships can enhance intelligence in a wide range of applications. However, one key problem is the missing of adaptation to heterogeneous on-device sensors. These low-end sensors on IoT devices possess different accuracies, granularities, and amounts of information, whose sensing qualities are heterogeneous and vary over time. The existing deep learning frameworks for IoT applications usually treat every sensor input equally over time or increase model capacity in an ad-hoc manner, lacking the ability to identify and exploit the sensor heterogeneities. In this work, we propose SADeepSense, a deep learning framework that can automatically balance the contributions of multiple sensor inputs over time by exploiting their sensing qualities. SADeepSense makes two key contributions. First, SADeepSense employs the self-attention mechanism to learn the correlations among different sensors over time with no additional supervision. The correlations are then applied to infer the sensing qualities and to reassign model concentrations in multiple sensors over time. Second, instead of directly learning the sensing qualities and contributions, SADeepSense generates the residual concentrations that are deviated from the equal contributions, which helps to stabilize the training process. We demonstrate the effectiveness of SADeepSense with two representative IoT sensing tasks: heterogeneous human activity recognition with motion sensors and gesture recognition with the wireless signal. SADeepSense consistently outperforms the state-of-the-art methods by a clear margin. In addition, we show that SADeepSense only imposes little additional resource-consumption burden on embedded devices compared to the corresponding state-of-the-art framework.

UR - http://www.scopus.com/inward/record.url?scp=85068212564&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068212564&partnerID=8YFLogxK

U2 - 10.1109/INFOCOM.2019.8737500

DO - 10.1109/INFOCOM.2019.8737500

M3 - Conference contribution

AN - SCOPUS:85068212564

T3 - Proceedings - IEEE INFOCOM

SP - 1243

EP - 1251

BT - INFOCOM 2019 - IEEE Conference on Computer Communications

PB - Institute of Electrical and Electronics Engineers Inc.

ER -