A study of Poisson query generation model for information retrieval

Qiaozhu Mei, Hui Fang, Chengxiang Zhai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Many variants of language models have been proposed for information retrieval. Most existing models are based on multinomial distribution and would score documents based on query likelihood computed based on a query generation probabilistic model. In this paper, we propose and study a new family of query generation models based on Poisson distribution. We show that while in their simplest forms, the new family of models and the existing multinomial models are equivalent. However, based on different smoothing methods, the two families of models behave differently. We show that the Poisson model has several advantages, including naturally accommodating per-term smoothing and modeling accurate background more efficiently. We present several variants of the new model corresponding to different smoothing methods, and evaluate them on four representative TREC test collections. The results show that while their basic models perform comparably, the Poisson model can out perform multinomial model with per-term smoothing. The performance can be further improved with two-stage smoothing.

Original languageEnglish (US)
Title of host publicationProceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR'07
Pages319-326
Number of pages8
DOIs
StatePublished - Nov 30 2007
Event30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR'07 - Amsterdam, Netherlands
Duration: Jul 23 2007Jul 27 2007

Publication series

NameProceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR'07

Other

Other30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR'07
Country/TerritoryNetherlands
CityAmsterdam
Period7/23/077/27/07

Keywords

  • Formal models
  • Poisson process
  • Query generation
  • Term dependent smoothing

ASJC Scopus subject areas

  • Information Systems
  • Software
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'A study of Poisson query generation model for information retrieval'. Together they form a unique fingerprint.

Cite this