"Be careful; Things can be worse than they appear" - Understanding biased algorithms and users' behavior around them in rating platforms

Motahhare Eslami, Kristen Vaccaro, Karrie Karahalios, Kevin Hamilton

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Awareness of bias in algorithms is growing among scholars and users of algorithmic systems. But what can we observe about how users discover and behave around such biases? We used a cross-platform audit technique that analyzed online ratings of 803 hotels across three hotel rating platforms and found that one site's algorithmic rating system biased ratings, particularly low-to-medium quality hotels, significantly higher than others (up to 37%). Analyzing reviews of 162 users who independently discovered this bias, we seek to understand if, how, and in what ways users perceive and manage this bias. Users changed the typical ways they used a review on a hotel rating platform to instead discuss the rating system itself and raise other users' awareness of the rating bias. This raising of awareness included practices like efforts to reverseengineer the rating algorithm, efforts to correct the bias, and demonstrations of broken trust. We conclude with a discussion of how such behavior patterns might inform design approaches that anticipate unexpected bias and provide reliable means for meaningful bias discovery and response.

Original languageEnglish (US)
Title of host publicationProceedings of the 11th International Conference on Web and Social Media, ICWSM 2017
PublisherAAAI Press
Pages62-71
Number of pages10
ISBN (Electronic)9781577357889
StatePublished - Jan 1 2017
Event11th International Conference on Web and Social Media, ICWSM 2017 - Montreal, Canada
Duration: May 15 2017May 18 2017

Publication series

NameProceedings of the 11th International Conference on Web and Social Media, ICWSM 2017

Other

Other11th International Conference on Web and Social Media, ICWSM 2017
CountryCanada
CityMontreal
Period5/15/175/18/17

Fingerprint

Hotels
Demonstrations

Keywords

  • Algorithm Audits
  • Algorithm Awareness
  • Algorithm Bias
  • Rating Platforms
  • Situated Action

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this

Eslami, M., Vaccaro, K., Karahalios, K., & Hamilton, K. (2017). "Be careful; Things can be worse than they appear" - Understanding biased algorithms and users' behavior around them in rating platforms. In Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017 (pp. 62-71). (Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017). AAAI Press.

"Be careful; Things can be worse than they appear" - Understanding biased algorithms and users' behavior around them in rating platforms. / Eslami, Motahhare; Vaccaro, Kristen; Karahalios, Karrie; Hamilton, Kevin.

Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017. AAAI Press, 2017. p. 62-71 (Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Eslami, M, Vaccaro, K, Karahalios, K & Hamilton, K 2017, "Be careful; Things can be worse than they appear" - Understanding biased algorithms and users' behavior around them in rating platforms. in Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017. Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017, AAAI Press, pp. 62-71, 11th International Conference on Web and Social Media, ICWSM 2017, Montreal, Canada, 5/15/17.
Eslami M, Vaccaro K, Karahalios K, Hamilton K. "Be careful; Things can be worse than they appear" - Understanding biased algorithms and users' behavior around them in rating platforms. In Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017. AAAI Press. 2017. p. 62-71. (Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017).
Eslami, Motahhare ; Vaccaro, Kristen ; Karahalios, Karrie ; Hamilton, Kevin. / "Be careful; Things can be worse than they appear" - Understanding biased algorithms and users' behavior around them in rating platforms. Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017. AAAI Press, 2017. pp. 62-71 (Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017).
@inproceedings{44530b04b6bc4d7f93bfaa8e4d733989,
title = "{"}Be careful; Things can be worse than they appear{"} - Understanding biased algorithms and users' behavior around them in rating platforms",
abstract = "Awareness of bias in algorithms is growing among scholars and users of algorithmic systems. But what can we observe about how users discover and behave around such biases? We used a cross-platform audit technique that analyzed online ratings of 803 hotels across three hotel rating platforms and found that one site's algorithmic rating system biased ratings, particularly low-to-medium quality hotels, significantly higher than others (up to 37{\%}). Analyzing reviews of 162 users who independently discovered this bias, we seek to understand if, how, and in what ways users perceive and manage this bias. Users changed the typical ways they used a review on a hotel rating platform to instead discuss the rating system itself and raise other users' awareness of the rating bias. This raising of awareness included practices like efforts to reverseengineer the rating algorithm, efforts to correct the bias, and demonstrations of broken trust. We conclude with a discussion of how such behavior patterns might inform design approaches that anticipate unexpected bias and provide reliable means for meaningful bias discovery and response.",
keywords = "Algorithm Audits, Algorithm Awareness, Algorithm Bias, Rating Platforms, Situated Action",
author = "Motahhare Eslami and Kristen Vaccaro and Karrie Karahalios and Kevin Hamilton",
year = "2017",
month = "1",
day = "1",
language = "English (US)",
series = "Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017",
publisher = "AAAI Press",
pages = "62--71",
booktitle = "Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017",

}

TY - GEN

T1 - "Be careful; Things can be worse than they appear" - Understanding biased algorithms and users' behavior around them in rating platforms

AU - Eslami, Motahhare

AU - Vaccaro, Kristen

AU - Karahalios, Karrie

AU - Hamilton, Kevin

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Awareness of bias in algorithms is growing among scholars and users of algorithmic systems. But what can we observe about how users discover and behave around such biases? We used a cross-platform audit technique that analyzed online ratings of 803 hotels across three hotel rating platforms and found that one site's algorithmic rating system biased ratings, particularly low-to-medium quality hotels, significantly higher than others (up to 37%). Analyzing reviews of 162 users who independently discovered this bias, we seek to understand if, how, and in what ways users perceive and manage this bias. Users changed the typical ways they used a review on a hotel rating platform to instead discuss the rating system itself and raise other users' awareness of the rating bias. This raising of awareness included practices like efforts to reverseengineer the rating algorithm, efforts to correct the bias, and demonstrations of broken trust. We conclude with a discussion of how such behavior patterns might inform design approaches that anticipate unexpected bias and provide reliable means for meaningful bias discovery and response.

AB - Awareness of bias in algorithms is growing among scholars and users of algorithmic systems. But what can we observe about how users discover and behave around such biases? We used a cross-platform audit technique that analyzed online ratings of 803 hotels across three hotel rating platforms and found that one site's algorithmic rating system biased ratings, particularly low-to-medium quality hotels, significantly higher than others (up to 37%). Analyzing reviews of 162 users who independently discovered this bias, we seek to understand if, how, and in what ways users perceive and manage this bias. Users changed the typical ways they used a review on a hotel rating platform to instead discuss the rating system itself and raise other users' awareness of the rating bias. This raising of awareness included practices like efforts to reverseengineer the rating algorithm, efforts to correct the bias, and demonstrations of broken trust. We conclude with a discussion of how such behavior patterns might inform design approaches that anticipate unexpected bias and provide reliable means for meaningful bias discovery and response.

KW - Algorithm Audits

KW - Algorithm Awareness

KW - Algorithm Bias

KW - Rating Platforms

KW - Situated Action

UR - http://www.scopus.com/inward/record.url?scp=85029439701&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029439701&partnerID=8YFLogxK

M3 - Conference contribution

T3 - Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017

SP - 62

EP - 71

BT - Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017

PB - AAAI Press

ER -