Utility data annotation with Amazon Mechanical Turk

Alexander Sorokin, David Alexander Forsyth

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We show how to outsource data annotation to Amazon Mechanical Turk. Doing so has produced annotations in quite large numbers relatively cheaply. The quality is good, and can be checked and controlled. Annotations are produced quickly. We describe results for several different annotation problems. We describe some strategies for determining when the task is well specified and properly priced.

Original languageEnglish (US)
Title of host publication2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops
DOIs
StatePublished - Sep 24 2008
Event2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops - Anchorage, AK, United States
Duration: Jun 23 2008Jun 28 2008

Publication series

Name2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops

Other

Other2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops
CountryUnited States
CityAnchorage, AK
Period6/23/086/28/08

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Cite this

Sorokin, A., & Forsyth, D. A. (2008). Utility data annotation with Amazon Mechanical Turk. In 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops [4562953] (2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops). https://doi.org/10.1109/CVPRW.2008.4562953