CrowdScreen: Algorithms for filtering data with humans

Aditya G Parameswaran, Hector Garcia-Molina, Hyunjung Park, Neoklis Polyzotis, Aditya Ramesh, Jennifer Widom

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Given a large set of data items, we consider the problem of filtering them based on a set of properties that can be verified by humans. This problem is commonplace in crowdsourcing applications, and yet, to our knowledge, no one has considered the formal optimization of this problem. (Typical solutions use heuristics to solve the problem.) We formally state a few different variants of this problem. We develop deterministic and probabilistic algorithms to optimize the expected cost (i.e., number of questions) and expected error. We experimentally show that our algorithms provide definite gains with respect to other strategies. Our algorithms can be applied in a variety of crowdsourcing scenarios and can form an integral part of any query processor that uses human computation.

Original languageEnglish (US)
Title of host publicationSIGMOD '12 - Proceedings of the International Conference on Management of Data
Pages361-372
Number of pages12
DOIs
StatePublished - Jun 28 2012
Externally publishedYes
Event2012 ACM SIGMOD International Conference on Management of Data, SIGMOD '12 - Scottsdale, AZ, United States
Duration: May 21 2012May 24 2012

Publication series

NameProceedings of the ACM SIGMOD International Conference on Management of Data
ISSN (Print)0730-8078

Other

Other2012 ACM SIGMOD International Conference on Management of Data, SIGMOD '12
Country/TerritoryUnited States
CityScottsdale, AZ
Period5/21/125/24/12

Keywords

  • crowdsourcing
  • filtering
  • human computation
  • predicates

ASJC Scopus subject areas

  • Software
  • Information Systems

Fingerprint

Dive into the research topics of 'CrowdScreen: Algorithms for filtering data with humans'. Together they form a unique fingerprint.

Cite this