Self-supervised role learning for graph neural networks

Aravind Sankar, Junting Wang, Adit Krishnan, Hari Sundaram

Research output: Contribution to journalArticlepeer-review

Abstract

We present InfoMotif, a new semi-supervised, motif-regularized, learning framework over graphs. We overcome two key limitations of message passing in popular graph neural networks (GNNs): localization (a k-layer GNN cannot utilize features outside the k-hop neighborhood of the labeled training nodes) and over-smoothed (structurally indistinguishable) representations. We formulate attributed structural roles of nodes based on their occurrence in different network motifs, independent of network proximity. Network motifs are higher-order structures indicating connectivity patterns between nodes and are crucial to the organization of complex networks. Two nodes share attributed structural roles if they participate in topologically similar motif instances over covarying sets of attributes. InfoMotif achieves architecture-agnostic regularization of arbitrary GNNs through novel self-supervised learning objectives based on mutual information maximization. Our training curriculum dynamically prioritizes multiple motifs in the learning process without relying on distributional assumptions in the underlying graph or the learning task. We integrate three state-of-the-art GNNs in our framework, to show notable performance gains (3–10% accuracy) across nine diverse real-world datasets spanning homogeneous and heterogeneous networks. Notably, we see stronger gains for nodes with sparse training labels and diverse attributes in local neighborhood structures.

Original languageEnglish (US)
Pages (from-to)2091-2121
Number of pages31
JournalKnowledge and Information Systems
Volume64
Issue number8
DOIs
StatePublished - Aug 2022
Externally publishedYes

Keywords

  • Association rule
  • Data mining
  • Itemset
  • Transaction collection

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Human-Computer Interaction
  • Hardware and Architecture
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Self-supervised role learning for graph neural networks'. Together they form a unique fingerprint.

Cite this