TY - GEN
T1 - Structuring, aggregating, and evaluating crowdsourced design critique
AU - Luther, Kurt
AU - Tolentino, Jari Lee
AU - Wu, Wei
AU - Pavel, Amy
AU - Bailey, Brian P.
AU - Agrawala, Maneesh
AU - Hartmann, Björn
AU - Dow, Steven P.
N1 - Funding Information:
National Science Foundation under IIS grants 1210836, 1208382, and 1217096.
PY - 2015/2/28
Y1 - 2015/2/28
N2 - Feedback is an important component of the design process, but gaining access to high-quality critique outside a classroom or firm is challenging. We present CrowdCrit, a webbased system that allows designers to receive design critiques from non-expert crowd workers. We evaluated CrowdCrit in three studies focusing on the designer's experience and benefits of the critiques. In the first study, we compared crowd and expert critiques and found evidence that aggregated crowd critique approaches expert critique. In a second study, we found that designers who got crowd feedback perceived that it improved their design process. The third study showed that designers were enthusiastic about crowd critiques and used them to change their designs. We conclude with implications for the design of crowd feedback services.
AB - Feedback is an important component of the design process, but gaining access to high-quality critique outside a classroom or firm is challenging. We present CrowdCrit, a webbased system that allows designers to receive design critiques from non-expert crowd workers. We evaluated CrowdCrit in three studies focusing on the designer's experience and benefits of the critiques. In the first study, we compared crowd and expert critiques and found evidence that aggregated crowd critique approaches expert critique. In a second study, we found that designers who got crowd feedback perceived that it improved their design process. The third study showed that designers were enthusiastic about crowd critiques and used them to change their designs. We conclude with implications for the design of crowd feedback services.
KW - Design; critique; feedback; social computing; crowdsourcing
UR - http://www.scopus.com/inward/record.url?scp=84968747313&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84968747313&partnerID=8YFLogxK
U2 - 10.1145/2675133.2675283
DO - 10.1145/2675133.2675283
M3 - Conference contribution
AN - SCOPUS:84968747313
T3 - CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing
SP - 473
EP - 485
BT - CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing
PB - Association for Computing Machinery, Inc
T2 - 18th ACM International Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2015
Y2 - 14 March 2015 through 18 March 2015
ER -