Structuring, aggregating, and evaluating crowdsourced design critique

Kurt Luther, Jari Lee Tolentino, Wei Wu, Amy Pavel, Brian P. Bailey, Maneesh Agrawala, Björn Hartmann, Steven P. Dow

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Feedback is an important component of the design process, but gaining access to high-quality critique outside a classroom or firm is challenging. We present CrowdCrit, a webbased system that allows designers to receive design critiques from non-expert crowd workers. We evaluated CrowdCrit in three studies focusing on the designer's experience and benefits of the critiques. In the first study, we compared crowd and expert critiques and found evidence that aggregated crowd critique approaches expert critique. In a second study, we found that designers who got crowd feedback perceived that it improved their design process. The third study showed that designers were enthusiastic about crowd critiques and used them to change their designs. We conclude with implications for the design of crowd feedback services.

Original languageEnglish (US)
Title of host publicationCSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing
PublisherAssociation for Computing Machinery, Inc
Pages473-485
Number of pages13
ISBN (Electronic)9781450329224
DOIs
StatePublished - Feb 28 2015
Event18th ACM International Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2015 - BC, Canada
Duration: Mar 14 2015Mar 18 2015

Publication series

NameCSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing

Other

Other18th ACM International Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2015
CountryCanada
CityBC
Period3/14/153/18/15

Keywords

  • Design; critique; feedback; social computing; crowdsourcing

ASJC Scopus subject areas

  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Structuring, aggregating, and evaluating crowdsourced design critique'. Together they form a unique fingerprint.

  • Cite this

    Luther, K., Tolentino, J. L., Wu, W., Pavel, A., Bailey, B. P., Agrawala, M., Hartmann, B., & Dow, S. P. (2015). Structuring, aggregating, and evaluating crowdsourced design critique. In CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing (pp. 473-485). (CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing). Association for Computing Machinery, Inc. https://doi.org/10.1145/2675133.2675283