Neural Network Coupled Model for Conversion and Exploitation of Heterogeneous Lexical Annotations

doi:10.13209/j.0479-8023.2019.098

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2020, Vol. 56 ›› Issue (1): 97-104.DOI: 10.13209/j.0479-8023.2019.098

Previous Articles Next Articles

Neural Network Coupled Model for Conversion and Exploitation of Heterogeneous Lexical Annotations

HUANG Depeng, LI Zhenghua^†, GONG Chen, ZHANG Min

School of Computer Science and Technology, Soochow University, Suzhou 215006

Received:2019-05-22 Revised:2019-09-19 Online:2020-01-20 Published:2020-01-20
Contact: LI Zhenghua, E-mail: zhli13(at)suda.edu.cn

基于神经耦合模型的异构词法数据转化和融合

黄德朋, 李正华^†, 龚晨, 张民

苏州大学计算机科学与技术学院, 苏州 215006

通讯作者: 李正华, E-mail: zhli13(at)suda.edu.cn
基金资助:
国家自然科学基金(61525205, 61876116, 61702518)和江苏高校优势学科建设工程项目资助

Abstract

Abstract:

In order to expand the scale of manual annotated data and thereby improve model performance, we attempt to make full use of existing heterogeneous annotations to learn model parameters. We extend coupled sequence labeling model proposed by Li et al. (2015) under the BiLSTM-based deep learning framework. The neural coupled model learn its parameters directly on two heterogeneous training data, and predicts two optimal sequences simultaneously during the test phase. A lot of experiments have been conducted on the part-of-speech (POS) tagging task and the joint word segmentation and POS (WS&POS) tagging task. The results show that neural coupled approach is superior to other methods for exploiting heterogeneous lexical data, including the multi-task learning method and the traditional discrete-feature coupled model. Neural coupled model achieves higher performance on both scenarios, i.e., annotation conversion and boost the final target-side tagging accuracy by exploiting heterogeneous data.

Key words: coupled model, BiLSTM, deep learning, part-of-speech tagging, word segmentation

摘要：

为了扩大人工标注数据的规模, 从而提高模型性能, 尝试充分利用已有的异构人工标注数据训练模型参数。将Li等2015年提出的耦合序列标注方法扩展到基于BiLSTM的深度学习框架, 直接在两个异构训练数据上训练参数, 测试阶段则同时预测两个标签序列。在词性标注、分词词性联合标注两个任务上进行大量实验, 结果表明, 与多任务学习方法和传统耦合模型相比, 神经耦合模型在利用词法异构数据方面更优越,在异构数据转化和融合两个场景上都取得更高的性能。

关键词: 耦合模型, BiLSTM, 深度学习, 词性标注, 分词

HUANG Depeng, LI Zhenghua, GONG Chen, ZHANG Min. Neural Network Coupled Model for Conversion and Exploitation of Heterogeneous Lexical Annotations[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 97-104.

黄德朋, 李正华, 龚晨, 张民. 基于神经耦合模型的异构词法数据转化和融合[J]. 北京大学学报自然科学版, 2020, 56(1): 97-104.

[1]	LI Fang, CAO Jian, LI Pu, XIE Hao, ZHAO Xiongbo, WANG Yuan, ZHANG Xing. Design and Implementation of Object Detection Acceleration Module Based on an ARM+FPGA Heterogeneous Platform [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(6): 1035-1041.
[2]	LI Tianran, LIU Mingtong, ZHANG Yujie, XU Jin’an, CHEN Yufeng. A Review of Entity Linking Research Based on Deep Learning [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 91-98.
[3]	HUANG Hongshi, WANG Zhengfei, XU Guoxiong, LI Wenxin, ZHANG Si, ZHANG Dongxia, AO Yingfang. Anterior Cruciate Ligament Deficiency Auxiliary Diagnosis Based on Plantar Pressure Information during Walking [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(5): 859-864.
[4]	FANG Haiquan, JIANG Yunzhong, YE Yuntao, CAO Yin. River Extraction from High-Resolution Satellite Images Combining Deep Learning and Multiple Chessboard Segmentation [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(4): 692-698.
[5]	CAI Zhenyu, GE Zengxi. Using Artificial Intelligence to Pick P-Wave First-Arrival of the Microseisms: Taking the Aftershock Sequence of Wenchuan Earthquake as an Example [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(3): 451-460.
[6]	WANG Qiansheng, YU Nan, ZHANG Meishan, HAN Zijia, FU Guohong. N3LDG: A Lightweight Neural Network Library for Natural Language Processing [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(1): 113-119.
[7]	LI Xiaojun, SHI Hanxiao, CHEN Nannan, LIU Hong, ZOU Yi. Research on Sentiment Analysis Based on Representation Learning [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(1): 105-112.
[8]	LUO Fan, WANG Houfeng. Chinese Text Sentiment Classification by H-RNN-CNN [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2018, 54(3): 459-465.
[9]	WU Huanqin, ZHANG Hongyang, LI Jingmei, ZHU Junguo, YANG Muyun, LI Sheng. Training Machine Translation Quality Estimation Model Based on Pseudo Data [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2018, 54(2): 279-285.
[10]	DU Liping, LI Xiaoge, YU Gen, LIU Chunli, LIU Rui. New Word Detection Based on an Improved PMI Algorithm for Enhancing Segmentation System [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2016, 52(1): 35-40.
[11]	SUN Daoxun;YANG Haijun. Hysteresis Effect in the Temperature and Precipitation Changes in Numerical Simulation with Varying Greenhouse Gas [J]. , 2015, 51(4): 763-771.
[12]	LIU Kan,YUAN Yunying. Short Texts Feature Extraction and Clustering Based on Auto-Encoder [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2015, 51(2): 282-288.
[13]	XI Xuefeng,ZHOU Guodong. Pronoun Resolution Based on Deep Learning [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2014, 50(1): 100-110.
[14]	RAO Gaoqi,XIU Chi,XUN Endong. Natural Annotation Research in Large-Scale Corpora with a Focus on Chinese Word Segmentation [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2013, 49(1): 140-146.
[15]	YUE Jinyuan,XU Jin’an,ZHANG Yujie. Chinese Word Segmentation for Patent Documents [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2013, 49(1): 159-164.

Neural Network Coupled Model for Conversion and Exploitation of Heterogeneous Lexical Annotations

基于神经耦合模型的异构词法数据转化和融合

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics