TY - JOUR
T1 - MEC-Enabled Hierarchical Emotion Recognition and Perturbation-Aware Defense in Smart Cities
AU - Zhao, Yi
AU - Xu, Ke
AU - Wang, Haiyang
AU - Li, Bo
AU - Qiao, Meina
AU - Shi, Haobin
N1 - Funding Information:
This work was supported in part by the China National Funds for Distinguished Young Scientists under Grant 61825204; in part by NSFC Project under Grant 61932016; in part by Beijing Outstanding Young Scientist Program under Grant BJJWZYJH01201910003011; in part by the National Key R&D Program of China under Grant 2018YFB0803405; in part by Beijing National Research Center for Information Science and Technology (BNRist) under Grant BNR2019RC01011; and in part by PCL Future Greater- Bay Area Network Facilities for Largescale Experiments and Applications under Grant LZC0019.
Publisher Copyright:
© 2014 IEEE.
PY - 2021/12/1
Y1 - 2021/12/1
N2 - With the explosive growth of Internet of Things (IoT) devices and various emerging network technologies, IoT-enabled smart cities are further refined into health smart cities. For example, IoT devices can automatically recognize emotional states through collected facial expressions, which can further serve mental health assessment, human-computer interaction, etc. On the other hand, existing facial expression recognition algorithms emphasize the application of deep neural networks (DNNs), and it is difficult for resource-constrained IoT devices to provide sufficient computing resources to optimize parameters for DNN-based structures. To solve the challenge of resource constraints, we propose the hierarchical emotion recognition system enabled by mobile edge computing (MEC). Specifically, MEC nodes provide IoT devices with short-delay and high-performance computing services, satisfying the requirements of training DNN-based algorithms. Moreover, our proposed emotion recognition system leverages a pretrained feature extraction module on the remote cloud to accelerate optimization and provides a localization module for specific tasks of IoT devices. In addition to evaluating the accuracy and efficiency, we also clarify that the DNN-based emotion recognition system exposes obvious vulnerability to perturbation. Due to the uncertainty of the environment, it is common for facial expressions collected by IoT devices to be accompanied by perturbation. To address this issue, we propose the proactive perturbation-aware defense mechanism. It has been demonstrated that the newly proposed defense mechanism can maintain state-of-the-art performance on the publicly available LIRIS-CSE dataset while defending against known and unknown perturbation. This can promote the deployment of our proposed MEC-enabled hierarchical emotion recognition system and defense mechanism in real-world scenarios.
AB - With the explosive growth of Internet of Things (IoT) devices and various emerging network technologies, IoT-enabled smart cities are further refined into health smart cities. For example, IoT devices can automatically recognize emotional states through collected facial expressions, which can further serve mental health assessment, human-computer interaction, etc. On the other hand, existing facial expression recognition algorithms emphasize the application of deep neural networks (DNNs), and it is difficult for resource-constrained IoT devices to provide sufficient computing resources to optimize parameters for DNN-based structures. To solve the challenge of resource constraints, we propose the hierarchical emotion recognition system enabled by mobile edge computing (MEC). Specifically, MEC nodes provide IoT devices with short-delay and high-performance computing services, satisfying the requirements of training DNN-based algorithms. Moreover, our proposed emotion recognition system leverages a pretrained feature extraction module on the remote cloud to accelerate optimization and provides a localization module for specific tasks of IoT devices. In addition to evaluating the accuracy and efficiency, we also clarify that the DNN-based emotion recognition system exposes obvious vulnerability to perturbation. Due to the uncertainty of the environment, it is common for facial expressions collected by IoT devices to be accompanied by perturbation. To address this issue, we propose the proactive perturbation-aware defense mechanism. It has been demonstrated that the newly proposed defense mechanism can maintain state-of-the-art performance on the publicly available LIRIS-CSE dataset while defending against known and unknown perturbation. This can promote the deployment of our proposed MEC-enabled hierarchical emotion recognition system and defense mechanism in real-world scenarios.
KW - Emotion Recognition
KW - Emotion recognition
KW - Facial Expression
KW - Internet of Things
KW - Internet of Things (IoT)
KW - Mobile Edge Computing (MEC)
KW - Performance evaluation
KW - Perturbation methods
KW - Proactive Perturbation-Aware Defense
KW - Robustness
KW - Robustness.
KW - Smart cities
KW - Visualization
KW - facial expression
KW - robustness
KW - mobile edge computing (MEC)
KW - proactive perturbation-aware defense
UR - http://www.scopus.com/inward/record.url?scp=85105891477&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85105891477&partnerID=8YFLogxK
U2 - 10.1109/JIOT.2021.3079304
DO - 10.1109/JIOT.2021.3079304
M3 - Article
AN - SCOPUS:85105891477
SN - 2327-4662
VL - 8
SP - 16933
EP - 16945
JO - IEEE Internet of Things Journal
JF - IEEE Internet of Things Journal
IS - 23
ER -