Embedding text-rich graph neural networks with sequence and topical semantic structures

Graph neural networks (GNNs) have demonstrated great power in tackling various analytical tasks on graph (i.e. network) data. However, graphs in the real world are usually text-rich, implying that valuable semantic structures need to be considered carefully. Existing GNNs for text-rich networks typically treat the text as attribute words alone, which inevitably leads to the loss of important semantic structures, limiting the representation capability of GNNs. To solve this limitation, we propose AS-GNN, an end-to-end adaptive GNN architecture via unified modelling of semantic structure and network propagation on text-rich networks. Specifically, we utilize semantic structure modelling part to capture both the local word-sequence and the global topic semantic structures from text. We then augment the original text-rich network into a tri-typed heterogeneous network (including document nodes, word nodes, and topic nodes) and accordingly design a semantic-aware propagation of information by introducing a discriminative convolutional mechanism. We further train these two parts together by leveraging distribution sharing and joint training strategies, so as to adaptively generate an appropriate network structure aiming at the learning objectives. In addition, we present a simplified semantic architecture S-GNN, which adopts the cascaded “Structure-GNN” pattern, to promote the efficiency of the model and be easily combined with existing GNNs. Extensive experiments on text-rich networks demonstrate the superiority of our new architectures over state of the arts. Meanwhile, such architectures can also be applied to e-commerce search scenes, and experiments on a real e-commerce problem from JD further illustrate the effectiveness of AS-GNN over the baselines.