MentorGNN: Deriving Curriculum for Pre-Training GNNs

Dawei Zhou, Lecheng Zheng, Dongqi Fu, Jiawei Han, Jingrui He

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Graph pre-training strategies have been attracting a surge of attention in the graph mining community, due to their flexibility in parameterizing graph neural networks (GNNs) without any label information. The key idea lies in encoding valuable information into the backbone GNNs, by predicting the masked graph signals extracted from the input graphs. In order to balance the importance of diverse graph signals (e.g., nodes, edges, subgraphs), the existing approaches are mostly hand-engineered by introducing hyperparameters to re-weight the importance of graph signals. However, human interventions with sub-optimal hyperparameters often inject additional bias and deteriorate the generalization performance in the downstream applications. This paper addresses these limitations from a new perspective, i.e., deriving curriculum for pre-training GNNs. We propose an end-to-end model named MentorGNN that aims to supervise the pre-training process of GNNs across graphs with diverse structures and disparate feature spaces. To comprehend heterogeneous graph signals at different granularities, we propose a curriculum learning paradigm that automatically re-weighs graph signals in order to ensure a good generalization in the target domain. Moreover, we shed new light on the problem of domain adaption on relational data (i.e., graphs) by deriving a natural and interpretable upper bound on the generalization error of the pre-trained GNNs. Extensive experiments on a wealth of real graphs validate and verify the performance of MentorGNN.

Original languageEnglish (US)
Title of host publicationCIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management
PublisherAssociation for Computing Machinery
Number of pages11
ISBN (Electronic)9781450392365
StatePublished - Oct 17 2022
Event31st ACM International Conference on Information and Knowledge Management, CIKM 2022 - Atlanta, United States
Duration: Oct 17 2022Oct 21 2022

Publication series

NameInternational Conference on Information and Knowledge Management, Proceedings


Conference31st ACM International Conference on Information and Knowledge Management, CIKM 2022
Country/TerritoryUnited States


  • domain adaptation
  • gnns
  • pre-training strategies

ASJC Scopus subject areas

  • General Business, Management and Accounting
  • General Decision Sciences


Dive into the research topics of 'MentorGNN: Deriving Curriculum for Pre-Training GNNs'. Together they form a unique fingerprint.

Cite this