Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification

Yu Zhang, Zhihong Shen, Chieh Han Wu, Boya Xie, Junheng Hao, Ye Yi Wang, Kuansan Wang, Jiawei Han

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Large-scale multi-label text classification (LMTC) aims to associate a document with its relevant labels from a large candidate set. Most existing LMTC approaches rely on massive human-annotated training data, which are often costly to obtain and suffer from a long-tailed label distribution (i.e., many labels occur only a few times in the training set). In this paper, we study LMTC under the zero-shot setting, which does not require any annotated documents with labels and only relies on label surface names and descriptions. To train a classifier that calculates the similarity score between a document and a label, we propose a novel metadata-induced contrastive learning (MICoL) method. Different from previous text-based contrastive learning techniques, MICoL exploits document metadata (e.g., authors, venues, and references of research papers), which are widely available on the Web, to derive similar document-document pairs. Experimental results on two large-scale datasets show that: (1) MICoL significantly outperforms strong zero-shot text classification and contrastive learning baselines; (2) MICoL is on par with the state-of-the-art supervised metadata-aware LMTC method trained on 10K-200K labeled documents; and (3) MICoL tends to predict more infrequent labels than supervised methods, thus alleviates the deteriorated performance on long-tailed labels.

Original languageEnglish (US)
Title of host publicationWWW 2022 - Proceedings of the ACM Web Conference 2022
PublisherAssociation for Computing Machinery
Number of pages12
ISBN (Electronic)9781450390965
StatePublished - Apr 25 2022
Event31st ACM World Wide Web Conference, WWW 2022 - Virtual, Online, France
Duration: Apr 25 2022Apr 29 2022

Publication series

NameWWW 2022 - Proceedings of the ACM Web Conference 2022


Conference31st ACM World Wide Web Conference, WWW 2022
CityVirtual, Online


  • contrastive learning
  • metadata
  • multi-label text classification

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software


Dive into the research topics of 'Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification'. Together they form a unique fingerprint.

Cite this