Document language models, query models, and risk minimization for information retrieval

J. Lafferty, C. Zhai

Research output: Contribution to journalConference articlepeer-review

Abstract

We present a framework for information retrieval that combines document models and query models using a probabilistic ranking function based on Bayesian decision theory. The framework suggests an operational retrieval model that extends recent developments in the language modelling approach to information retrieval. A language model for each document is estimated, as well as a language model for each query, and the retrieval problem is cast in terms of risk minimization. The query language model can be exploited to model user preferences, the context of a query, synonomy and word senses. While recent work has incorporated word translation models for this purpose, we introduce a new method using Markov chains defined on a set of documents to estimate the query models. The Markov chain method has connections to algorithms from link analysis and social networks. The new approach is evaluated on TREC collections and compared to the basic language modelling approach and vector space models together with query expansion using Rocchio. Significant improvements are obtained over standard query expansion methods for strong baseline TF-IDF systems, with the greatest improvements attained for short queries on Web data.

Original languageEnglish (US)
Pages (from-to)111-119
Number of pages9
JournalSIGIR Forum (ACM Special Interest Group on Information Retrieval)
DOIs
StatePublished - 2001
Externally publishedYes
Event24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval - New Orleans, LA, United States
Duration: Sep 9 2001Sep 13 2001

ASJC Scopus subject areas

  • Management Information Systems
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Document language models, query models, and risk minimization for information retrieval'. Together they form a unique fingerprint.

Cite this