On scalability and robustness limitations of real and asymptotic confidence bounds in social sensing

Dong Wang, Lance Kaplan, Tarek Abdelzaher, Charu C. Aggarwal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper estimates new confidence bounds on source reliability in social sensing applications. Scalable and robust estimation of source reliability is a key challenge in social sensing where humans or human-operated sensors act as data sources. In order to assess correctness of data, the reliability of sources must first be assessed, yet this is complicated when sources are not a priori known and vetted, but rather can opt in at will, for example, by downloading a sensing application on their mobile device. In our previous work, we developed a maximum likelihood source reliability estimator and approximately quantified confidence in its estimation based on an asymptotic Cramer-Rao lower bound (CRLB). In this paper we show that the asymptotic bound fails to track estimation performance when the number of sources is small. We derive the real CRLB to accurately characterize estimation performance for scenarios where the asymptotic bound fails. We study the limitations of the real and asymptotic CRLBs and show the trade-offs they offer between computational complexity and estimation scalability. We also evaluate the robustness of these bounds to changes in the number of sources. The results offer an understanding of attainable estimation accuracy of source reliability in social sensing applications that rely on un-vetted sources whose reliability is not known in advance.

Original languageEnglish (US)
Title of host publication2012 9th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks, SECON 2012
Pages506-514
Number of pages9
DOIs
StatePublished - 2012
Event2012 9th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks, SECON 2012 - Seoul, Korea, Republic of
Duration: Jun 18 2012Jun 21 2012

Publication series

NameAnnual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks workshops
Volume1
ISSN (Print)2155-5486
ISSN (Electronic)2155-5494

Other

Other2012 9th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks, SECON 2012
CountryKorea, Republic of
CitySeoul
Period6/18/126/21/12

Keywords

  • CRLB
  • Quantification
  • Robustness
  • Scalability
  • Social Sensing

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'On scalability and robustness limitations of real and asymptotic confidence bounds in social sensing'. Together they form a unique fingerprint.

Cite this