Understanding workers, developing effective tasks, and enhancing marketplace dynamics: A study of a large crowdsourcing marketplace

Ayush Jain, Akash Das Sarma, Aditya Parameswaran, Jennifer Widom

Research output: Contribution to journalConference article

Abstract

We conduct an experimental analysis of a dataset comprising over 27 million microtasks performed by over 70,000 workers issued to a large crowdsourcing marketplace between 2012-2016. Using this data-never before analyzed in an academic context-we shed light on three crucial aspects of crowdsourcing: (1) Task design -helping requesters understand what constitutes an effective task, and how to go about designing one; (2) Marketplace dynamics - helping marketplace administrators and designers understand the interaction between tasks and workers, and the corresponding marketplace load; and (3) Worker behavior - understanding worker attention spans, lifetimes, and general behavior, for the improvement of the crowdsourcing ecosystem as a whole.

Original languageEnglish (US)
Pages (from-to)829-840
Number of pages12
JournalProceedings of the VLDB Endowment
Volume10
Issue number7
DOIs
StatePublished - Jan 1 2017
Event43rd International Conference on Very Large Data Bases, VLDB 2017 - Munich, Germany
Duration: Aug 28 2017Sep 1 2017

Fingerprint

Ecosystems

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Cite this

Understanding workers, developing effective tasks, and enhancing marketplace dynamics : A study of a large crowdsourcing marketplace. / Jain, Ayush; Das Sarma, Akash; Parameswaran, Aditya; Widom, Jennifer.

In: Proceedings of the VLDB Endowment, Vol. 10, No. 7, 01.01.2017, p. 829-840.

Research output: Contribution to journalConference article

Jain, Ayush ; Das Sarma, Akash ; Parameswaran, Aditya ; Widom, Jennifer. / Understanding workers, developing effective tasks, and enhancing marketplace dynamics : A study of a large crowdsourcing marketplace. In: Proceedings of the VLDB Endowment. 2017 ; Vol. 10, No. 7. pp. 829-840.
@article{41db3ee5d8dc49999770b7b0e545b077,
title = "Understanding workers, developing effective tasks, and enhancing marketplace dynamics: A study of a large crowdsourcing marketplace",
abstract = "We conduct an experimental analysis of a dataset comprising over 27 million microtasks performed by over 70,000 workers issued to a large crowdsourcing marketplace between 2012-2016. Using this data-never before analyzed in an academic context-we shed light on three crucial aspects of crowdsourcing: (1) Task design -helping requesters understand what constitutes an effective task, and how to go about designing one; (2) Marketplace dynamics - helping marketplace administrators and designers understand the interaction between tasks and workers, and the corresponding marketplace load; and (3) Worker behavior - understanding worker attention spans, lifetimes, and general behavior, for the improvement of the crowdsourcing ecosystem as a whole.",
author = "Ayush Jain and {Das Sarma}, Akash and Aditya Parameswaran and Jennifer Widom",
year = "2017",
month = "1",
day = "1",
doi = "10.14778/3067421.3067431",
language = "English (US)",
volume = "10",
pages = "829--840",
journal = "Proceedings of the VLDB Endowment",
issn = "2150-8097",
publisher = "Very Large Data Base Endowment Inc.",
number = "7",

}

TY - JOUR

T1 - Understanding workers, developing effective tasks, and enhancing marketplace dynamics

T2 - A study of a large crowdsourcing marketplace

AU - Jain, Ayush

AU - Das Sarma, Akash

AU - Parameswaran, Aditya

AU - Widom, Jennifer

PY - 2017/1/1

Y1 - 2017/1/1

N2 - We conduct an experimental analysis of a dataset comprising over 27 million microtasks performed by over 70,000 workers issued to a large crowdsourcing marketplace between 2012-2016. Using this data-never before analyzed in an academic context-we shed light on three crucial aspects of crowdsourcing: (1) Task design -helping requesters understand what constitutes an effective task, and how to go about designing one; (2) Marketplace dynamics - helping marketplace administrators and designers understand the interaction between tasks and workers, and the corresponding marketplace load; and (3) Worker behavior - understanding worker attention spans, lifetimes, and general behavior, for the improvement of the crowdsourcing ecosystem as a whole.

AB - We conduct an experimental analysis of a dataset comprising over 27 million microtasks performed by over 70,000 workers issued to a large crowdsourcing marketplace between 2012-2016. Using this data-never before analyzed in an academic context-we shed light on three crucial aspects of crowdsourcing: (1) Task design -helping requesters understand what constitutes an effective task, and how to go about designing one; (2) Marketplace dynamics - helping marketplace administrators and designers understand the interaction between tasks and workers, and the corresponding marketplace load; and (3) Worker behavior - understanding worker attention spans, lifetimes, and general behavior, for the improvement of the crowdsourcing ecosystem as a whole.

UR - http://www.scopus.com/inward/record.url?scp=85021626555&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021626555&partnerID=8YFLogxK

U2 - 10.14778/3067421.3067431

DO - 10.14778/3067421.3067431

M3 - Conference article

AN - SCOPUS:85021626555

VL - 10

SP - 829

EP - 840

JO - Proceedings of the VLDB Endowment

JF - Proceedings of the VLDB Endowment

SN - 2150-8097

IS - 7

ER -