Curriculum learning for heterogeneous star network embedding via deep reinforcement learning

Meng Qu, Jian Tang, Jiawei Han

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Learning node representations for networks has attracted much attention recently due to its effectiveness in a variety of applications. This paper focuses on learning node representations for heterogeneous star networks, which have a center node type linked with multiple attribute node types through different types of edges. In heterogeneous star networks, we observe that the training order of different types of edges affects the learning performance significantly. Therefore we study learning curricula for node representation learning in heterogeneous star networks, i.e., learning an optimal sequence of edges of different types for the node representation learning process. We formulate the problem as a Markov decision process, with the action as selecting a specific type of edges for learning or terminating the training process, and the state as the sequence of edge types selected so far. The reward is calculated as the performance on external tasks with node representations as features, and the goal is to take a series of actions to maximize the cumulative rewards. We propose an approach based on deep reinforcement learning for this problem. Our approach leverages LSTM models to encode states and further estimate the expected cumulative reward of each state-action pair, which essentially measures the long-term performance of different actions at each state. Experimental results on real-world heterogeneous star networks demonstrate the effectiveness and efficiency of our approach over competitive baseline approaches.

Original languageEnglish (US)
Title of host publicationWSDM 2018 - Proceedings of the 11th ACM International Conference on Web Search and Data Mining
PublisherAssociation for Computing Machinery, Inc
Pages468-476
Number of pages9
ISBN (Electronic)9781450355810
DOIs
StatePublished - Feb 2 2018
Event11th ACM International Conference on Web Search and Data Mining, WSDM 2018 - Marina Del Rey, United States
Duration: Feb 5 2018Feb 9 2018

Publication series

NameWSDM 2018 - Proceedings of the 11th ACM International Conference on Web Search and Data Mining
Volume2018-Febuary

Other

Other11th ACM International Conference on Web Search and Data Mining, WSDM 2018
CountryUnited States
CityMarina Del Rey
Period2/5/182/9/18

ASJC Scopus subject areas

  • Computer Science Applications
  • Software
  • Computer Networks and Communications
  • Information Systems

Fingerprint Dive into the research topics of 'Curriculum learning for heterogeneous star network embedding via deep reinforcement learning'. Together they form a unique fingerprint.

  • Cite this

    Qu, M., Tang, J., & Han, J. (2018). Curriculum learning for heterogeneous star network embedding via deep reinforcement learning. In WSDM 2018 - Proceedings of the 11th ACM International Conference on Web Search and Data Mining (pp. 468-476). (WSDM 2018 - Proceedings of the 11th ACM International Conference on Web Search and Data Mining; Vol. 2018-Febuary). Association for Computing Machinery, Inc. https://doi.org/10.1145/3159652.3159711