A classroom study of using crowd feedback in the iterative design process

Anbang Xu, Huaming Rao, Steven P. Dow, Brian P Bailey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Crowd feedback systems offer designers an emerging approach for improving their designs, but there is little empirical evidence of the benefit of these systems. This paper reports the results of a study of using a crowd feedback system to iterate on visual designs. Users in an introductory visual design course created initial designs satisfying a design brief and received crowd feedback on the designs. Users revised the designs and the system was used to generate feedback again. This format enabled us to detect the changes between the initial and revised designs and how the feedback related to those changes. Further, we analyzed the value of crowd feedback by comparing it with expert evaluation and feedback generated via free-form prompts. Results showed that the crowd feedback system prompted deep and cosmetic changes and led to improved designs, the crowd recognized the design improvements, and structured workflows generated more interpretative, diverse and critical feedback than free-form prompts.

Original languageEnglish (US)
Title of host publicationCSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing
PublisherAssociation for Computing Machinery, Inc
Pages1637-1648
Number of pages12
ISBN (Electronic)9781450329224
DOIs
StatePublished - Feb 28 2015
Event18th ACM International Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2015 - BC, Canada
Duration: Mar 14 2015Mar 18 2015

Publication series

NameCSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing

Other

Other18th ACM International Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2015
CountryCanada
CityBC
Period3/14/153/18/15

Fingerprint

Feedback

Keywords

  • Crowd feedback
  • creativity.
  • crowdsourcing
  • design iteration
  • visual design

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this

Xu, A., Rao, H., Dow, S. P., & Bailey, B. P. (2015). A classroom study of using crowd feedback in the iterative design process. In CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing (pp. 1637-1648). (CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing). Association for Computing Machinery, Inc. https://doi.org/10.1145/2675133.2675140

A classroom study of using crowd feedback in the iterative design process. / Xu, Anbang; Rao, Huaming; Dow, Steven P.; Bailey, Brian P.

CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing. Association for Computing Machinery, Inc, 2015. p. 1637-1648 (CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Xu, A, Rao, H, Dow, SP & Bailey, BP 2015, A classroom study of using crowd feedback in the iterative design process. in CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing. CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing, Association for Computing Machinery, Inc, pp. 1637-1648, 18th ACM International Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2015, BC, Canada, 3/14/15. https://doi.org/10.1145/2675133.2675140
Xu A, Rao H, Dow SP, Bailey BP. A classroom study of using crowd feedback in the iterative design process. In CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing. Association for Computing Machinery, Inc. 2015. p. 1637-1648. (CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing). https://doi.org/10.1145/2675133.2675140
Xu, Anbang ; Rao, Huaming ; Dow, Steven P. ; Bailey, Brian P. / A classroom study of using crowd feedback in the iterative design process. CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing. Association for Computing Machinery, Inc, 2015. pp. 1637-1648 (CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing).
@inproceedings{57d3f7f09e924d049b33a69d0f471a1b,
title = "A classroom study of using crowd feedback in the iterative design process",
abstract = "Crowd feedback systems offer designers an emerging approach for improving their designs, but there is little empirical evidence of the benefit of these systems. This paper reports the results of a study of using a crowd feedback system to iterate on visual designs. Users in an introductory visual design course created initial designs satisfying a design brief and received crowd feedback on the designs. Users revised the designs and the system was used to generate feedback again. This format enabled us to detect the changes between the initial and revised designs and how the feedback related to those changes. Further, we analyzed the value of crowd feedback by comparing it with expert evaluation and feedback generated via free-form prompts. Results showed that the crowd feedback system prompted deep and cosmetic changes and led to improved designs, the crowd recognized the design improvements, and structured workflows generated more interpretative, diverse and critical feedback than free-form prompts.",
keywords = "Crowd feedback, creativity., crowdsourcing, design iteration, visual design",
author = "Anbang Xu and Huaming Rao and Dow, {Steven P.} and Bailey, {Brian P}",
year = "2015",
month = "2",
day = "28",
doi = "10.1145/2675133.2675140",
language = "English (US)",
series = "CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing",
publisher = "Association for Computing Machinery, Inc",
pages = "1637--1648",
booktitle = "CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing",

}

TY - GEN

T1 - A classroom study of using crowd feedback in the iterative design process

AU - Xu, Anbang

AU - Rao, Huaming

AU - Dow, Steven P.

AU - Bailey, Brian P

PY - 2015/2/28

Y1 - 2015/2/28

N2 - Crowd feedback systems offer designers an emerging approach for improving their designs, but there is little empirical evidence of the benefit of these systems. This paper reports the results of a study of using a crowd feedback system to iterate on visual designs. Users in an introductory visual design course created initial designs satisfying a design brief and received crowd feedback on the designs. Users revised the designs and the system was used to generate feedback again. This format enabled us to detect the changes between the initial and revised designs and how the feedback related to those changes. Further, we analyzed the value of crowd feedback by comparing it with expert evaluation and feedback generated via free-form prompts. Results showed that the crowd feedback system prompted deep and cosmetic changes and led to improved designs, the crowd recognized the design improvements, and structured workflows generated more interpretative, diverse and critical feedback than free-form prompts.

AB - Crowd feedback systems offer designers an emerging approach for improving their designs, but there is little empirical evidence of the benefit of these systems. This paper reports the results of a study of using a crowd feedback system to iterate on visual designs. Users in an introductory visual design course created initial designs satisfying a design brief and received crowd feedback on the designs. Users revised the designs and the system was used to generate feedback again. This format enabled us to detect the changes between the initial and revised designs and how the feedback related to those changes. Further, we analyzed the value of crowd feedback by comparing it with expert evaluation and feedback generated via free-form prompts. Results showed that the crowd feedback system prompted deep and cosmetic changes and led to improved designs, the crowd recognized the design improvements, and structured workflows generated more interpretative, diverse and critical feedback than free-form prompts.

KW - Crowd feedback

KW - creativity.

KW - crowdsourcing

KW - design iteration

KW - visual design

UR - http://www.scopus.com/inward/record.url?scp=84944690311&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84944690311&partnerID=8YFLogxK

U2 - 10.1145/2675133.2675140

DO - 10.1145/2675133.2675140

M3 - Conference contribution

AN - SCOPUS:84944690311

T3 - CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing

SP - 1637

EP - 1648

BT - CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing

PB - Association for Computing Machinery, Inc

ER -