Leveraging comparable corpora for cross-lingual information retrieval in resource-lean language pairs

Azadeh Shakery, Cheng Xiang Zhai

Research output: Contribution to journalArticlepeer-review

Abstract

Cross-language information retrieval (CLIR) has so far been studied with the assumption that some rich linguistic resources such as bilingual dictionaries or parallel corpora are available. But creation of such high quality resources is labor-intensive and they are not always at hand. In this paper we investigate the feasibility of using only comparable corpora for CLIR, without relying on other linguistic resources. Comparable corpora are text documents in different languages that cover similar topics and are often naturally attainable (e. g., news articles published in different languages at the same time period). We adapt an existing cross-lingual word association mining method and incorporate it into a language modeling approach to cross-language retrieval. We investigate different strategies for estimating the target query language models. Our evaluation results on the TREC Arabic-English cross-lingual data show that the proposed method is effective for the CLIR task, demonstrating that it is feasible to perform cross-lingual information retrieval with just comparable corpora.

Original languageEnglish (US)
Pages (from-to)1-29
Number of pages29
JournalInformation Retrieval
Volume16
Issue number1
DOIs
StatePublished - Feb 2013

Keywords

  • Comparable corpora
  • Cross-language information retrieval
  • Language models
  • Probabilistic propagation

ASJC Scopus subject areas

  • Information Systems
  • Library and Information Sciences

Fingerprint

Dive into the research topics of 'Leveraging comparable corpora for cross-lingual information retrieval in resource-lean language pairs'. Together they form a unique fingerprint.

Cite this