Analysis of adaptive training for learning to rank in information retrieval

Saar Kuzi, Sahiti Labhishetty, Shubhra Kanti Karmaker Santu, Prasad Pradip Joshi, Cheng Xiang Zhai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Learning to Rank is an important framework used in search engines to optimize the combination of multiple features in a single ranking function. In the existing work on learning to rank, such a ranking function is often trained on a large set of different queries to optimize the overall performance on all of them. However, the optimal parameters to combine those features are generally query-dependent, making such a strategy of “one size fits all” non-optimal. Some previous works have addressed this problem by suggesting a query-level adaptive training for learning to rank with promising results. However, previous work has not analyzed the reasons for the improvement. In this paper, we present a Best-Feature Calibration (BFC) strategy for analyzing learning to rank models and use this strategy to examine the benefit of query-level adaptive training. Our results show that the benefit of adaptive training mainly lies in the improvement of the robustness of learning to rank in cases where it does not perform as well as the best single feature.

Original languageEnglish (US)
Title of host publicationCIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management
PublisherAssociation for Computing Machinery
Pages2325-2328
Number of pages4
ISBN (Electronic)9781450369763
DOIs
StatePublished - Nov 3 2019
Event28th ACM International Conference on Information and Knowledge Management, CIKM 2019 - Beijing, China
Duration: Nov 3 2019Nov 7 2019

Publication series

NameInternational Conference on Information and Knowledge Management, Proceedings

Conference

Conference28th ACM International Conference on Information and Knowledge Management, CIKM 2019
CountryChina
CityBeijing
Period11/3/1911/7/19

Fingerprint

Information retrieval
Learning to rank
Query
Ranking function
Calibration
Robustness
Search engine

ASJC Scopus subject areas

  • Business, Management and Accounting(all)
  • Decision Sciences(all)

Cite this

Kuzi, S., Labhishetty, S., Santu, S. K. K., Joshi, P. P., & Zhai, C. X. (2019). Analysis of adaptive training for learning to rank in information retrieval. In CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management (pp. 2325-2328). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3357384.3358159

Analysis of adaptive training for learning to rank in information retrieval. / Kuzi, Saar; Labhishetty, Sahiti; Santu, Shubhra Kanti Karmaker; Joshi, Prasad Pradip; Zhai, Cheng Xiang.

CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2019. p. 2325-2328 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kuzi, S, Labhishetty, S, Santu, SKK, Joshi, PP & Zhai, CX 2019, Analysis of adaptive training for learning to rank in information retrieval. in CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, pp. 2325-2328, 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, 11/3/19. https://doi.org/10.1145/3357384.3358159
Kuzi S, Labhishetty S, Santu SKK, Joshi PP, Zhai CX. Analysis of adaptive training for learning to rank in information retrieval. In CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery. 2019. p. 2325-2328. (International Conference on Information and Knowledge Management, Proceedings). https://doi.org/10.1145/3357384.3358159
Kuzi, Saar ; Labhishetty, Sahiti ; Santu, Shubhra Kanti Karmaker ; Joshi, Prasad Pradip ; Zhai, Cheng Xiang. / Analysis of adaptive training for learning to rank in information retrieval. CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2019. pp. 2325-2328 (International Conference on Information and Knowledge Management, Proceedings).
@inproceedings{3381d29eab35490bb743cec23dbd6196,
title = "Analysis of adaptive training for learning to rank in information retrieval",
abstract = "Learning to Rank is an important framework used in search engines to optimize the combination of multiple features in a single ranking function. In the existing work on learning to rank, such a ranking function is often trained on a large set of different queries to optimize the overall performance on all of them. However, the optimal parameters to combine those features are generally query-dependent, making such a strategy of “one size fits all” non-optimal. Some previous works have addressed this problem by suggesting a query-level adaptive training for learning to rank with promising results. However, previous work has not analyzed the reasons for the improvement. In this paper, we present a Best-Feature Calibration (BFC) strategy for analyzing learning to rank models and use this strategy to examine the benefit of query-level adaptive training. Our results show that the benefit of adaptive training mainly lies in the improvement of the robustness of learning to rank in cases where it does not perform as well as the best single feature.",
author = "Saar Kuzi and Sahiti Labhishetty and Santu, {Shubhra Kanti Karmaker} and Joshi, {Prasad Pradip} and Zhai, {Cheng Xiang}",
year = "2019",
month = "11",
day = "3",
doi = "10.1145/3357384.3358159",
language = "English (US)",
series = "International Conference on Information and Knowledge Management, Proceedings",
publisher = "Association for Computing Machinery",
pages = "2325--2328",
booktitle = "CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management",

}

TY - GEN

T1 - Analysis of adaptive training for learning to rank in information retrieval

AU - Kuzi, Saar

AU - Labhishetty, Sahiti

AU - Santu, Shubhra Kanti Karmaker

AU - Joshi, Prasad Pradip

AU - Zhai, Cheng Xiang

PY - 2019/11/3

Y1 - 2019/11/3

N2 - Learning to Rank is an important framework used in search engines to optimize the combination of multiple features in a single ranking function. In the existing work on learning to rank, such a ranking function is often trained on a large set of different queries to optimize the overall performance on all of them. However, the optimal parameters to combine those features are generally query-dependent, making such a strategy of “one size fits all” non-optimal. Some previous works have addressed this problem by suggesting a query-level adaptive training for learning to rank with promising results. However, previous work has not analyzed the reasons for the improvement. In this paper, we present a Best-Feature Calibration (BFC) strategy for analyzing learning to rank models and use this strategy to examine the benefit of query-level adaptive training. Our results show that the benefit of adaptive training mainly lies in the improvement of the robustness of learning to rank in cases where it does not perform as well as the best single feature.

AB - Learning to Rank is an important framework used in search engines to optimize the combination of multiple features in a single ranking function. In the existing work on learning to rank, such a ranking function is often trained on a large set of different queries to optimize the overall performance on all of them. However, the optimal parameters to combine those features are generally query-dependent, making such a strategy of “one size fits all” non-optimal. Some previous works have addressed this problem by suggesting a query-level adaptive training for learning to rank with promising results. However, previous work has not analyzed the reasons for the improvement. In this paper, we present a Best-Feature Calibration (BFC) strategy for analyzing learning to rank models and use this strategy to examine the benefit of query-level adaptive training. Our results show that the benefit of adaptive training mainly lies in the improvement of the robustness of learning to rank in cases where it does not perform as well as the best single feature.

UR - http://www.scopus.com/inward/record.url?scp=85075451594&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075451594&partnerID=8YFLogxK

U2 - 10.1145/3357384.3358159

DO - 10.1145/3357384.3358159

M3 - Conference contribution

AN - SCOPUS:85075451594

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 2325

EP - 2328

BT - CIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management

PB - Association for Computing Machinery

ER -