TY - JOUR
T1 - Adversarial Learning for Multi-Task Sequence Labeling with Attention Mechanism
AU - Wang, Yu
AU - Li, Yun
AU - Zhu, Ziye
AU - Tong, Hanghang
AU - Huang, Yue
N1 - Funding Information:
This work was partially supported by Natural Science Foundation of China (No. 61772284), Postgraduate Research & Practice Innovation Program of Jiangsu Province (SJKY19-0766). Hanghang Tong author is partially supported by Natural Science Foundation (1947135, 2003924 and 1939725).
Funding Information:
Manuscript received October 12, 2019; revised March 7, 2020, April 21, 2020, and July 9, 2020; accepted July 15, 2020. Date of publication July 30, 2020; date of current version September 3, 2020. This work was partially supported by Natural Science Foundation of China (No. 61772284), Postgraduate Research & Practice Innovation Program of Jiangsu Province (SJKY19_0766). Hanghang Tong author is partially supported by Natural Science Foundation (1947135, 2003924 and 1939725). The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Jianfeng Gao. (Corresponding author: Yun Li.) Yu Wang, Yun Li, Ziye Zhu, and Yue Huang are with the Jiangsu Key Laboratory of Big Data Security and Intelligent Processing, Nanjing University of Posts and Telecommunications, Nanjing, China (e-mail: 2017070114@njupt.edu.cn; liyun@njupt.edu.cn; 1015041217@njupt.edu.cn; huangyue@njupt.edu.cn).
Publisher Copyright:
© 2014 IEEE.
PY - 2020
Y1 - 2020
N2 - With the requirements of natural language applications, multi-task sequence labeling methods have some immediate benefits over the single-task sequence labeling methods. Recently, many state-of-the-art multi-task sequence labeling methods were proposed, while still many issues to be resolved including (C1) exploring a more general relationship between tasks, (C2) extracting the task-shared knowledge purely and (C3) merging the task-shared knowledge for each task appropriately. To address the above challenges, we propose MTAA, a symmetric multi-task sequence labeling model, which performs an arbitrary number of tasks simultaneously. Furthermore, MTAA extracts the shared knowledge among tasks by adversarial learning and integrates the proposed multi-representation fusion attention mechanism for merging feature representations. We evaluate MTAA on two widely used data sets: CoNLL2003 and OntoNotes5.0. Experimental results show that our proposed model outperforms the latest methods on the named entity recognition and the syntactic chunking task by a large margin, and achieves state-of-the-art results on the part-of-speech tagging task.
AB - With the requirements of natural language applications, multi-task sequence labeling methods have some immediate benefits over the single-task sequence labeling methods. Recently, many state-of-the-art multi-task sequence labeling methods were proposed, while still many issues to be resolved including (C1) exploring a more general relationship between tasks, (C2) extracting the task-shared knowledge purely and (C3) merging the task-shared knowledge for each task appropriately. To address the above challenges, we propose MTAA, a symmetric multi-task sequence labeling model, which performs an arbitrary number of tasks simultaneously. Furthermore, MTAA extracts the shared knowledge among tasks by adversarial learning and integrates the proposed multi-representation fusion attention mechanism for merging feature representations. We evaluate MTAA on two widely used data sets: CoNLL2003 and OntoNotes5.0. Experimental results show that our proposed model outperforms the latest methods on the named entity recognition and the syntactic chunking task by a large margin, and achieves state-of-the-art results on the part-of-speech tagging task.
KW - Adversarial learning
KW - attention mechanism
KW - multi-task learning
KW - sequence labeling
UR - http://www.scopus.com/inward/record.url?scp=85091009101&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85091009101&partnerID=8YFLogxK
U2 - 10.1109/TASLP.2020.3013114
DO - 10.1109/TASLP.2020.3013114
M3 - Article
AN - SCOPUS:85091009101
SN - 2329-9290
VL - 28
SP - 2476
EP - 2488
JO - IEEE/ACM Transactions on Audio Speech and Language Processing
JF - IEEE/ACM Transactions on Audio Speech and Language Processing
M1 - 9153102
ER -