VIR Lab: A platform for privacy-preserving evaluation for information retrieval models

Hui Fang, Chengxiang Zhai

Research output: Contribution to journalArticlepeer-review

Abstract

Information retrieval (IR) has been a highly empirical discipline since the very beginning of the field. The development and study of any novel techniques such as retrieval models always require extensive experiments over multiple representative data collections. Traditionally, IR evaluation relies on the use of publicly available data, so researchers often download the collections and conduct the evaluation on their servers. However, this would not be a favorable (or even possible) solution to evaluation over the proprietary data due to various privacy concerns. In this paper, we discuss one potential solution to the privacy-preserving evaluation (PPE) for IR models. We first briefly introduce the VIRLab system, and then discuss how to extend the system to enable a controlled data-centric experimental environment for evaluation over proprietary data.

Original languageEnglish (US)
Pages (from-to)37-38
Number of pages2
JournalCEUR Workshop Proceedings
Volume1225
StatePublished - 2014

Keywords

  • Data-centric evaluation
  • PPE
  • Privacy-preserving evaluation
  • Virtual IR lab

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'VIR Lab: A platform for privacy-preserving evaluation for information retrieval models'. Together they form a unique fingerprint.

Cite this