VIR Lab: A platform for privacy-preserving evaluation for information retrieval models

Research output: Contribution to journalConference articlepeer-review


Information retrieval (IR) has been a highly empirical discipline since the very beginning of the field. The development and study of any novel techniques such as retrieval models always require extensive experiments over multiple representative data collections. Traditionally, IR evaluation relies on the use of publicly available data, so researchers often download the collections and conduct the evaluation on their servers. However, this would not be a favorable (or even possible) solution to evaluation over the proprietary data due to various privacy concerns. In this paper, we discuss one potential solution to the privacy-preserving evaluation (PPE) for IR models. We first briefly introduce the VIRLab system, and then discuss how to extend the system to enable a controlled data-centric experimental environment for evaluation over proprietary data.

Original languageEnglish (US)
Pages (from-to)37-38
Number of pages2
JournalCEUR Workshop Proceedings
StatePublished - 2014
Event1st International Workshop on Privacy-Preserving IR: When Information Retrieval Meets Privacy and Security, PIR 2014, Co-Located with 37th Annual International ACM SIGIR Conference, SIGIR 2014 - Gold Coast, Australia
Duration: Jul 11 2014Jul 11 2014


  • Data-centric evaluation
  • PPE
  • Privacy-preserving evaluation
  • Virtual IR lab

ASJC Scopus subject areas

  • General Computer Science


Dive into the research topics of 'VIR Lab: A platform for privacy-preserving evaluation for information retrieval models'. Together they form a unique fingerprint.

Cite this