Towards Privacy-Preserving Evaluation for Information Retrieval Models Over Industry Data Sets

Peilin Yang, Mianwei Zhou, Yi Chang, Chengxiang Zhai, Hui Fang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The development of Information Retrieval (IR) techniques heavily depends on empirical studies over real world data collections. Unfortunately, those real world data sets are often unavailable to researchers due to privacy concerns. In fact, the lack of publicly available industry data sets has become a serious bottleneck hindering IR research. To address this problem, we propose to bridge the gap between academic research and industry data sets through a privacy-preserving evaluation platform. The novelty of the platform lies in its “data-centric” mechanism, where the data sit on a secure server and IR algorithms to be evaluated would be uploaded to the server. The platform will run the codes of the algorithms and return the evaluation results. Preliminary experiments with retrieval models reveal interesting new observations and insights about state of the art retrieval models, demonstrating the value of an industry data set.

Original languageEnglish (US)
Title of host publicationInformation Retrieval Technology - 13th Asia Information Retrieval Societies Conference, AIRS 2017, Proceedings
EditorsGrace Hui Yang, Shuo Xu, Won-Kyung Sung, Hanmin Jung, Seungbock Lee, Jeonghoon Lee, Krisana Chinnasarn, Kazutoshi Sumiya, Zhicheng Dou, Young-Guk Ha
PublisherSpringer
Pages210-221
Number of pages12
ISBN (Print)9783319701448
DOIs
StatePublished - 2017
Event13th Asia Information Retrieval Societies Conference, AIRS 2017 - Jeju Island, Korea, Republic of
Duration: Nov 22 2017Nov 24 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10648 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other13th Asia Information Retrieval Societies Conference, AIRS 2017
Country/TerritoryKorea, Republic of
CityJeju Island
Period11/22/1711/24/17

Keywords

  • Evaluation
  • Privacy
  • Test collections

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Towards Privacy-Preserving Evaluation for Information Retrieval Models Over Industry Data Sets'. Together they form a unique fingerprint.

Cite this