Using multiple uery aspects to build test collections without human relevance udgments

Miles Efron

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Collecting relevance judgments (qrels) is an especially challenging part of building an information retrieval test collection. This paper presents a novel method for creating test collections by offering a substitute for relevance judgments. Our method is based on an old idea in IR: a single information need can be represented by many query articulations. We call different articulations of a particular need query aspects. By combining the top k documents retrieved by a single system for multiple query aspects, we build judgment-free qrels whose rank ordering of IR systems correlates highly with rankings based on human relevance judgments.

Original languageEnglish (US)
Title of host publicationAdvances in Information Retrieval - 31th European Conference on IR Research, ECIR 2009, Proceedings
Pages301-312
Number of pages12
DOIs
StatePublished - Jul 23 2009
Event31th European Conference on Information Retrieval, ECIR 2009 - Toulouse, France
Duration: Apr 6 2009Apr 9 2009

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5478 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other31th European Conference on Information Retrieval, ECIR 2009
CountryFrance
CityToulouse
Period4/6/094/9/09

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Using multiple uery aspects to build test collections without human relevance udgments'. Together they form a unique fingerprint.

Cite this