Multi-task Knowledge Graph Representations via Residual Functions

Adit Krishnan; Mahashweta Das; Mangesh Bendre; Fei Wang; Hao Yang; Hari Sundaram

doi:10.1007/978-3-031-05933-9_21

Multi-task Knowledge Graph Representations via Residual Functions

Adit Krishnan, Mahashweta Das, Mangesh Bendre, Fei Wang, Hao Yang, Hari Sundaram

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper, we propose MuTATE, a Multi-Task Augmented approach to learn Transferable Embeddings of knowledge graphs. Previous knowledge graph representation techniques either employ task-agnostic geometric hypotheses to learn informative node embeddings or integrate task-specific learning objectives like attribute prediction. In contrast, our framework unifies multiple co-dependent learning objectives with knowledge graph enrichment. We define co-dependence as multiple tasks that extract covariant distributions of entities and their relationships for prediction or regression objectives. We facilitate knowledge transfer in this setting: tasks → graph, graph → tasks, and task-1 → task-2 via task-specific residual functions to specialize the node embeddings for each task, motivated by domain-shift theory. We show 5% relative gains over state-of-the-art knowledge graph embedding baselines on two public multi-task datasets and show significant potential for cross-task learning.

Original language	English (US)
Title of host publication	Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings
Editors	João Gama, Tianrui Li, Yang Yu, Enhong Chen, Yu Zheng, Fei Teng
Publisher	Springer
Pages	262-275
Number of pages	14
ISBN (Print)	9783031059322
DOIs	https://doi.org/10.1007/978-3-031-05933-9_21
State	Published - 2022
Event	26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022 - Chengdu, China Duration: May 16 2022 → May 19 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13280 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022
Country/Territory	China
City	Chengdu
Period	5/16/22 → 5/19/22

Keywords

Graph neural networks
Knowledge graph embedding
Knowledge graphs
Multi-task learning
Residual learning

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Online availability

10.1007/978-3-031-05933-9_21

Library availability

Discover UIUC Full Text

Cite this

Krishnan, A., Das, M., Bendre, M., Wang, F., Yang, H., & Sundaram, H. (2022). Multi-task Knowledge Graph Representations via Residual Functions. In J. Gama, T. Li, Y. Yu, E. Chen, Y. Zheng, & F. Teng (Eds.), Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings (pp. 262-275). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13280 LNAI). Springer. https://doi.org/10.1007/978-3-031-05933-9_21

Multi-task Knowledge Graph Representations via Residual Functions. / Krishnan, Adit; Das, Mahashweta; Bendre, Mangesh et al.
Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings. ed. / João Gama; Tianrui Li; Yang Yu; Enhong Chen; Yu Zheng; Fei Teng. Springer, 2022. p. 262-275 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13280 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Krishnan, A, Das, M, Bendre, M, Wang, F, Yang, H & Sundaram, H 2022, Multi-task Knowledge Graph Representations via Residual Functions. in J Gama, T Li, Y Yu, E Chen, Y Zheng & F Teng (eds), Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13280 LNAI, Springer, pp. 262-275, 26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022, Chengdu, China, 5/16/22. https://doi.org/10.1007/978-3-031-05933-9_21

Krishnan A, Das M, Bendre M, Wang F, Yang H, Sundaram H. Multi-task Knowledge Graph Representations via Residual Functions. In Gama J, Li T, Yu Y, Chen E, Zheng Y, Teng F, editors, Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings. Springer. 2022. p. 262-275. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-05933-9_21

Krishnan, Adit ; Das, Mahashweta ; Bendre, Mangesh et al. / Multi-task Knowledge Graph Representations via Residual Functions. Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings. editor / João Gama ; Tianrui Li ; Yang Yu ; Enhong Chen ; Yu Zheng ; Fei Teng. Springer, 2022. pp. 262-275 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{4a60551780f44e968fd580a8f1358257,

title = "Multi-task Knowledge Graph Representations via Residual Functions",

abstract = "In this paper, we propose MuTATE, a Multi-Task Augmented approach to learn Transferable Embeddings of knowledge graphs. Previous knowledge graph representation techniques either employ task-agnostic geometric hypotheses to learn informative node embeddings or integrate task-specific learning objectives like attribute prediction. In contrast, our framework unifies multiple co-dependent learning objectives with knowledge graph enrichment. We define co-dependence as multiple tasks that extract covariant distributions of entities and their relationships for prediction or regression objectives. We facilitate knowledge transfer in this setting: tasks → graph, graph → tasks, and task-1 → task-2 via task-specific residual functions to specialize the node embeddings for each task, motivated by domain-shift theory. We show 5% relative gains over state-of-the-art knowledge graph embedding baselines on two public multi-task datasets and show significant potential for cross-task learning.",

keywords = "Graph neural networks, Knowledge graph embedding, Knowledge graphs, Multi-task learning, Residual learning",

author = "Adit Krishnan and Mahashweta Das and Mangesh Bendre and Fei Wang and Hao Yang and Hari Sundaram",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022 ; Conference date: 16-05-2022 Through 19-05-2022",

year = "2022",

doi = "10.1007/978-3-031-05933-9_21",

language = "English (US)",

isbn = "9783031059322",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "262--275",

editor = "Jo{\~a}o Gama and Tianrui Li and Yang Yu and Enhong Chen and Yu Zheng and Fei Teng",

booktitle = "Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings",

address = "Germany",

}

TY - GEN

T1 - Multi-task Knowledge Graph Representations via Residual Functions

AU - Krishnan, Adit

AU - Das, Mahashweta

AU - Bendre, Mangesh

AU - Wang, Fei

AU - Yang, Hao

AU - Sundaram, Hari

PY - 2022

Y1 - 2022

N2 - In this paper, we propose MuTATE, a Multi-Task Augmented approach to learn Transferable Embeddings of knowledge graphs. Previous knowledge graph representation techniques either employ task-agnostic geometric hypotheses to learn informative node embeddings or integrate task-specific learning objectives like attribute prediction. In contrast, our framework unifies multiple co-dependent learning objectives with knowledge graph enrichment. We define co-dependence as multiple tasks that extract covariant distributions of entities and their relationships for prediction or regression objectives. We facilitate knowledge transfer in this setting: tasks → graph, graph → tasks, and task-1 → task-2 via task-specific residual functions to specialize the node embeddings for each task, motivated by domain-shift theory. We show 5% relative gains over state-of-the-art knowledge graph embedding baselines on two public multi-task datasets and show significant potential for cross-task learning.

AB - In this paper, we propose MuTATE, a Multi-Task Augmented approach to learn Transferable Embeddings of knowledge graphs. Previous knowledge graph representation techniques either employ task-agnostic geometric hypotheses to learn informative node embeddings or integrate task-specific learning objectives like attribute prediction. In contrast, our framework unifies multiple co-dependent learning objectives with knowledge graph enrichment. We define co-dependence as multiple tasks that extract covariant distributions of entities and their relationships for prediction or regression objectives. We facilitate knowledge transfer in this setting: tasks → graph, graph → tasks, and task-1 → task-2 via task-specific residual functions to specialize the node embeddings for each task, motivated by domain-shift theory. We show 5% relative gains over state-of-the-art knowledge graph embedding baselines on two public multi-task datasets and show significant potential for cross-task learning.

KW - Graph neural networks

KW - Knowledge graph embedding

KW - Knowledge graphs

KW - Multi-task learning

KW - Residual learning

UR - http://www.scopus.com/inward/record.url?scp=85130347193&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85130347193&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-05933-9_21

DO - 10.1007/978-3-031-05933-9_21

M3 - Conference contribution

AN - SCOPUS:85130347193

SN - 9783031059322

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 262

EP - 275

BT - Advances in Knowledge Discovery and Data Mining - 26th Pacific-Asia Conference, PAKDD 2022, Proceedings

A2 - Gama, João

A2 - Li, Tianrui

A2 - Yu, Yang

A2 - Chen, Enhong

A2 - Zheng, Yu

A2 - Teng, Fei

PB - Springer

T2 - 26th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2022

Y2 - 16 May 2022 through 19 May 2022

ER -