Representing documents via latent keyphrase inference

Jialu Liu, Xiang Ren, Jingbo Shang, Taylor Cassidy, Clare R. Voss, Jiawei Han

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Many text mining approaches adopt bag-of-words or n-grams models to represent documents. Looking beyond just the words, i.e., the explicit surface forms, in a document can improve a computer's understanding of text. Being aware of this, researchers have proposed concept-based models that rely on a human-curated knowledge base to incorporate other related concepts in the document representation. But these methods are not desirable when applied to vertical domains (e.g., literature, enterprise, etc.) due to low coverage of in-domain concepts in the general knowledge base and interference from out-of-domain concepts. In this paper, we propose a data-driven model named Latent Keyphrase In-ference (LAKI ) that represents documents with a vector of closely related domain keyphrases instead of single words or existing concepts in the knowledge base. We show that given a corpus of in-domain documents, topical content units can be learned for each domain keyphrase, which enables a computer to do smart inference to discover latent document keyphrases, going beyond just explicit mentions. Compared with the state-of-Art document representation approaches, LAKI fills the gap between bag-of-words and concept-based models by using domain keyphrases as the basic representation unit. It removes dependency on a knowledge base while providing, with keyphrases, readily interpretable representations. When evaluated against 8 other methods on two text mining tasks over two corpora, LAKI outperformed all.

Original languageEnglish (US)
Title of host publication25th International World Wide Web Conference, WWW 2016
PublisherInternational World Wide Web Conferences Steering Committee
Number of pages11
ISBN (Electronic)9781450341431
StatePublished - 2016
Event25th International World Wide Web Conference, WWW 2016 - Montreal, Canada
Duration: Apr 11 2016Apr 15 2016

Publication series

Name25th International World Wide Web Conference, WWW 2016


Other25th International World Wide Web Conference, WWW 2016

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software


Dive into the research topics of 'Representing documents via latent keyphrase inference'. Together they form a unique fingerprint.

Cite this