Recursive Ground Truth Estimator for Social Data Streams

Shuochao Yao, Md Tanvir Amin, Lu Su, Shaohan Hu, Shen Li, Shiguang Wang, Yiran Zhao, Tarek Abdelzaher, Lance Kaplan, Charu Aggarwal, Aylin Yener

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The paper develops a recursive state estimator for social network data streams that allows exploitation of social networks, such as Twitter, as sensor networks to reliably observe physical events. Recent literature suggested using social networks as sensor networks leveraging the fact that much of the information upload on the former constitutes acts of sensing. A significant challenge identified in that context was that source reliability is often unknown, leading to uncertainty regarding the veracity of reported observations. Multiple truth finding systems were developed to solve this problem, generally geared towards batch analysis of offline datasets. This work complements the present batch approaches by developing an online recursive state estimator that recovers ground truth from streaming data. In this paper, we model physical world state by a set of binary signals (propositions, called assertions, about world state) and the social network as a noisy medium, where distortion, fabrication, omissions, and duplication are introduced. Our recursive state estimator is designed to recover the original binary signal (the true propositions) from the received noisy signal, essentially decoding the unreliable social network output to obtain the best estimate of ground truth in the physical world. Results show that the estimator is both effective and efficient at recovering the original signal with a high degree of accuracy. The estimator gives rise to a novel situation awareness tool that can be used for reliably following unfolding events in real time, using dynamically arriving social network data.

Original languageEnglish (US)
Title of host publication2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2016 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509008025
DOIs
StatePublished - Apr 26 2016
Event15th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2016 - Vienna, Austria
Duration: Apr 11 2016Apr 14 2016

Publication series

Name2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2016 - Proceedings

Other

Other15th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2016
CountryAustria
CityVienna
Period4/11/164/14/16

    Fingerprint

ASJC Scopus subject areas

  • Signal Processing
  • Computer Networks and Communications

Cite this

Yao, S., Amin, M. T., Su, L., Hu, S., Li, S., Wang, S., Zhao, Y., Abdelzaher, T., Kaplan, L., Aggarwal, C., & Yener, A. (2016). Recursive Ground Truth Estimator for Social Data Streams. In 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2016 - Proceedings [7460719] (2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2016 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPSN.2016.7460719