Exploiting thread structures to improve smoothing of language models for forum post retrieval

Huizhong Duan; Chengxiang Zhai

doi:10.1007/978-3-642-20161-5_35

Exploiting thread structures to improve smoothing of language models for forum post retrieval

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Due to many unique characteristics of forum data, forum post retrieval is different from traditional document retrieval and web search, raising interesting research questions about how to optimize the accuracy of forum post retrieval. In this paper, we study how to exploit the naturally available raw thread structures of forums to improve retrieval accuracy in the language modeling framework. Specifically, we propose and study two different schemes for smoothing the language model of a forum post based on the thread containing the post. We explore several different variants of the two schemes to exploit thread structures in different ways. We also create a human annotated test data set for forum post retrieval and evaluate the proposed smoothing methods using this data set. The experiment results show that the proposed methods for leveraging forum threads to improve estimation of document language models are effective, and they outperform the existing smoothing methods for the forum post retrieval task.

Original language	English (US)
Title of host publication	Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings
Editors	Paul Clough, Colum Foley, Cathal Gurrin, Hyowon Lee, Gareth J.F. Jones, Wessel Kraaij, Vanessa Murdoch
Publisher	Springer
Pages	350-361
Number of pages	12
ISBN (Print)	9783642201608
DOIs	https://doi.org/10.1007/978-3-642-20161-5_35
State	Published - 2011
Event	33rd European Conference on Information Retrieval, ECIR 2011 - Dublin, Ireland Duration: Apr 18 2011 → Apr 21 2011

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	6611 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	33rd European Conference on Information Retrieval, ECIR 2011
Country/Territory	Ireland
City	Dublin
Period	4/18/11 → 4/21/11

Keywords

Forum post retrieval
Language modeling
Smoothing

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Online availability

10.1007/978-3-642-20161-5_35

Library availability

Discover UIUC Full Text

Cite this

Duan, H., & Zhai, C. (2011). Exploiting thread structures to improve smoothing of language models for forum post retrieval. In P. Clough, C. Foley, C. Gurrin, H. Lee, G. J. F. Jones, W. Kraaij, & V. Murdoch (Eds.), Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings (pp. 350-361). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6611 LNCS). Springer. https://doi.org/10.1007/978-3-642-20161-5_35

Exploiting thread structures to improve smoothing of language models for forum post retrieval. / Duan, Huizhong; Zhai, Chengxiang.
Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings. ed. / Paul Clough; Colum Foley; Cathal Gurrin; Hyowon Lee; Gareth J.F. Jones; Wessel Kraaij; Vanessa Murdoch. Springer, 2011. p. 350-361 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6611 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Duan, H & Zhai, C 2011, Exploiting thread structures to improve smoothing of language models for forum post retrieval. in P Clough, C Foley, C Gurrin, H Lee, GJF Jones, W Kraaij & V Murdoch (eds), Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6611 LNCS, Springer, pp. 350-361, 33rd European Conference on Information Retrieval, ECIR 2011, Dublin, Ireland, 4/18/11. https://doi.org/10.1007/978-3-642-20161-5_35

Duan H, Zhai C. Exploiting thread structures to improve smoothing of language models for forum post retrieval. In Clough P, Foley C, Gurrin C, Lee H, Jones GJF, Kraaij W, Murdoch V, editors, Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings. Springer. 2011. p. 350-361. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-20161-5_35

Duan, Huizhong ; Zhai, Chengxiang. / Exploiting thread structures to improve smoothing of language models for forum post retrieval. Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings. editor / Paul Clough ; Colum Foley ; Cathal Gurrin ; Hyowon Lee ; Gareth J.F. Jones ; Wessel Kraaij ; Vanessa Murdoch. Springer, 2011. pp. 350-361 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b72d6eecc9504d719447d243a4795d0f,

title = "Exploiting thread structures to improve smoothing of language models for forum post retrieval",

abstract = "Due to many unique characteristics of forum data, forum post retrieval is different from traditional document retrieval and web search, raising interesting research questions about how to optimize the accuracy of forum post retrieval. In this paper, we study how to exploit the naturally available raw thread structures of forums to improve retrieval accuracy in the language modeling framework. Specifically, we propose and study two different schemes for smoothing the language model of a forum post based on the thread containing the post. We explore several different variants of the two schemes to exploit thread structures in different ways. We also create a human annotated test data set for forum post retrieval and evaluate the proposed smoothing methods using this data set. The experiment results show that the proposed methods for leveraging forum threads to improve estimation of document language models are effective, and they outperform the existing smoothing methods for the forum post retrieval task.",

keywords = "Forum post retrieval, Language modeling, Smoothing",

author = "Huizhong Duan and Chengxiang Zhai",

note = "Publisher Copyright: {\textcopyright} Springer-Verlag Berlin Heidelberg 2011.; 33rd European Conference on Information Retrieval, ECIR 2011 ; Conference date: 18-04-2011 Through 21-04-2011",

year = "2011",

doi = "10.1007/978-3-642-20161-5_35",

language = "English (US)",

isbn = "9783642201608",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "350--361",

editor = "Paul Clough and Colum Foley and Cathal Gurrin and Hyowon Lee and Jones, {Gareth J.F.} and Wessel Kraaij and Vanessa Murdoch",

booktitle = "Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings",

address = "Germany",

}

TY - GEN

T1 - Exploiting thread structures to improve smoothing of language models for forum post retrieval

AU - Duan, Huizhong

AU - Zhai, Chengxiang

N1 - Publisher Copyright: © Springer-Verlag Berlin Heidelberg 2011.

PY - 2011

Y1 - 2011

N2 - Due to many unique characteristics of forum data, forum post retrieval is different from traditional document retrieval and web search, raising interesting research questions about how to optimize the accuracy of forum post retrieval. In this paper, we study how to exploit the naturally available raw thread structures of forums to improve retrieval accuracy in the language modeling framework. Specifically, we propose and study two different schemes for smoothing the language model of a forum post based on the thread containing the post. We explore several different variants of the two schemes to exploit thread structures in different ways. We also create a human annotated test data set for forum post retrieval and evaluate the proposed smoothing methods using this data set. The experiment results show that the proposed methods for leveraging forum threads to improve estimation of document language models are effective, and they outperform the existing smoothing methods for the forum post retrieval task.

AB - Due to many unique characteristics of forum data, forum post retrieval is different from traditional document retrieval and web search, raising interesting research questions about how to optimize the accuracy of forum post retrieval. In this paper, we study how to exploit the naturally available raw thread structures of forums to improve retrieval accuracy in the language modeling framework. Specifically, we propose and study two different schemes for smoothing the language model of a forum post based on the thread containing the post. We explore several different variants of the two schemes to exploit thread structures in different ways. We also create a human annotated test data set for forum post retrieval and evaluate the proposed smoothing methods using this data set. The experiment results show that the proposed methods for leveraging forum threads to improve estimation of document language models are effective, and they outperform the existing smoothing methods for the forum post retrieval task.

KW - Forum post retrieval

KW - Language modeling

KW - Smoothing

UR - http://www.scopus.com/inward/record.url?scp=85025157437&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85025157437&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-20161-5_35

DO - 10.1007/978-3-642-20161-5_35

M3 - Conference contribution

AN - SCOPUS:85025157437

SN - 9783642201608

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 350

EP - 361

BT - Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings

A2 - Clough, Paul

A2 - Foley, Colum

A2 - Gurrin, Cathal

A2 - Lee, Hyowon

A2 - Jones, Gareth J.F.

A2 - Kraaij, Wessel

A2 - Murdoch, Vanessa

PB - Springer

T2 - 33rd European Conference on Information Retrieval, ECIR 2011

Y2 - 18 April 2011 through 21 April 2011

ER -

Exploiting thread structures to improve smoothing of language models for forum post retrieval

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Online availability

Library availability

Related links

Fingerprint

Cite this