Unsupervised rank aggregation with domain-specific expertise

Alexandre Klementiev, Dan Roth, Kevin Small, Ivan Titov

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Consider the setting where a panel of judges is repeatedly asked to (partially) rank sets of objects according to given criteria, and assume that the judges' expertise depends on the objects' domain. Learning to aggregate their rankings with the goal of producing a better joint ranking is a fundamental problem in many areas of Information Retrieval and Natural Language Processing, amongst others. However, supervised ranking data is generally difficult to obtain, especially if coming from multiple domains. Therefore, we propose a framework for learning to aggregate votes of constituent rankers with domain specific expertise without supervision. We apply the learning framework to the settings of aggregating full rankings and aggregating top-k lists, demonstrating significant improvements over a domain-agnostic baseline in both cases.

Original languageEnglish (US)
Title of host publicationIJCAI-09 - Proceedings of the 21st International Joint Conference on Artificial Intelligence
PublisherInternational Joint Conferences on Artificial Intelligence
Number of pages6
ISBN (Print)9781577354260
StatePublished - Jan 1 2009
Event21st International Joint Conference on Artificial Intelligence, IJCAI 2009 - Pasadena, United States
Duration: Jul 11 2009Jul 16 2009

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
ISSN (Print)1045-0823


Conference21st International Joint Conference on Artificial Intelligence, IJCAI 2009
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Artificial Intelligence


Dive into the research topics of 'Unsupervised rank aggregation with domain-specific expertise'. Together they form a unique fingerprint.

Cite this