Multiclass performance metric elicitation

Gaurush Hiranandani; Shant Boodaghians; Ruta Mehta; Oluwasanmi Oluseye Koyejo

Multiclass performance metric elicitation

Gaurush Hiranandani, Shant Boodaghians, Ruta Mehta, Oluwasanmi Oluseye Koyejo

Research output: Contribution to journal › Conference article › peer-review

Abstract

Metric Elicitation is a principled framework for selecting the performance metric that best reflects implicit user preferences. However, available strategies have so far been limited to binary classification. In this paper, we propose novel strategies for eliciting multiclass classification performance metrics using only relative preference feedback. We also show that the strategies are robust to both finite sample and feedback noise.

Original language	English (US)
Journal	Advances in Neural Information Processing Systems
Volume	32
State	Published - 2019
Event	33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019 - Vancouver, Canada Duration: Dec 8 2019 → Dec 14 2019

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems
Signal Processing

Access to Document

Fingerprint Dive into the research topics of 'Multiclass performance metric elicitation'. Together they form a unique fingerprint.

Cite this

@article{0db337aec6014c2e9c5347308c2e012f,

title = "Multiclass performance metric elicitation",

abstract = "Metric Elicitation is a principled framework for selecting the performance metric that best reflects implicit user preferences. However, available strategies have so far been limited to binary classification. In this paper, we propose novel strategies for eliciting multiclass classification performance metrics using only relative preference feedback. We also show that the strategies are robust to both finite sample and feedback noise.",

author = "Gaurush Hiranandani and Shant Boodaghians and Ruta Mehta and Koyejo, {Oluwasanmi Oluseye}",

year = "2019",

language = "English (US)",

volume = "32",

journal = "Advances in Neural Information Processing Systems",

issn = "1049-5258",

note = "33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019 ; Conference date: 08-12-2019 Through 14-12-2019",

}

TY - JOUR

T1 - Multiclass performance metric elicitation

AU - Hiranandani, Gaurush

AU - Boodaghians, Shant

AU - Mehta, Ruta

AU - Koyejo, Oluwasanmi Oluseye

PY - 2019

Y1 - 2019

N2 - Metric Elicitation is a principled framework for selecting the performance metric that best reflects implicit user preferences. However, available strategies have so far been limited to binary classification. In this paper, we propose novel strategies for eliciting multiclass classification performance metrics using only relative preference feedback. We also show that the strategies are robust to both finite sample and feedback noise.

AB - Metric Elicitation is a principled framework for selecting the performance metric that best reflects implicit user preferences. However, available strategies have so far been limited to binary classification. In this paper, we propose novel strategies for eliciting multiclass classification performance metrics using only relative preference feedback. We also show that the strategies are robust to both finite sample and feedback noise.

UR - http://www.scopus.com/inward/record.url?scp=85090171477&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85090171477&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85090171477

VL - 32

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

T2 - 33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019

Y2 - 8 December 2019 through 14 December 2019

ER -