A Sentence Segmentation Method for Ancient Chinese Texts Based on Recurrent Neural Network

doi:10.13209/j.0479-8023.2017.032

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2017, Vol. 53 ›› Issue (2): 255-261.DOI: 10.13209/j.0479-8023.2017.032

• Orginal Article • Previous Articles Next Articles

A Sentence Segmentation Method for Ancient Chinese Texts Based on Recurrent Neural Network

Boli WANG¹, Xiaodong SHI^1,^2,^3,^†(), Jinsong SU⁴

1. Department of Cognitive Science, Xiamen University, Xiamen 361005
2. Collaborative Innovation Center for Peaceful Development of Cross-Strait Relations, Xiamen University, Xiamen 361005
3. Fujian Province Key Laboratory for Brain-inspired Computing, Xiamen 361005
4. Software School, Xiamen University, Xiamen 361005

Received:2016-07-29 Revised:2016-10-07 Online:2016-11-30 Published:2017-03-20
Contact: Xiaodong SHI

一种基于循环神经网络的古文断句方法

王博立¹, 史晓东^1,^2,^3,^†(), 苏劲松⁴

1. 厦门大学智能科学与技术系, 厦门 361005
2. 厦门大学两岸关系和平发展协同创新中心, 厦门 361005
3. 福建省类脑计算技术及应用重点实验室, 厦门 361005
4. 厦门大学软件学院, 厦门 361005

通讯作者: 史晓东
基金资助:
教育部专项“简繁汉字智能转换系统”、国家科技支撑计划项目(2012BAH14F03)、教育部博士点基金(20130121110040)、国家自然科学基金(61573294)和CCF中文信息技术开放课题(CCF2015-01-01)资助

Abstract

Abstract:

This paper proposes an automatic sentence segmentation method for ancient Chinese texts based on recurrent neural network (RNN). A bi-directional RNN structure with gated recurrent units (GRU) is implemented, and state transition probability and length penalty are employed in decoding to improve the accuracy. Experimental results show that proposed model achieves higher F1 score than traditional methods.

Key words: ancient Chinese, sentence segmentation, recurrent neural network

摘要：

提出一种基于循环神经网络的古文自动断句方法。该方法采用基于GRU (gated recurrent unit)的双向循环神经网络进行古文断句。在解码过程中, 该算法不仅利用神经网络输出的概率分布, 还进一步引入状态转移概率和长度惩罚, 以便提高断句准确率。在大规模古籍语料上的实验结果表明, 所提方法能够取得比传统方法更高的断句F1值。

关键词: 古汉语, 断句, 循环神经网络

CLC Number:

TP391

Boli WANG, Xiaodong SHI, Jinsong SU. A Sentence Segmentation Method for Ancient Chinese Texts Based on Recurrent Neural Network[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2017, 53(2): 255-261.

王博立, 史晓东, 苏劲松. 一种基于循环神经网络的古文断句方法[J]. 北京大学学报自然科学版, 2017, 53(2): 255-261.

Figures/Tables 6

References 19

[1]	黄建年, 侯汉清. 农业古籍断句标点模式研究. 中文信息学报, 2008, 22(4): 31-38
[2]	陈天莹, 陈蓉, 潘璐璐, 等. 基于前后文 n-gram 模型的古汉语句子切分. 计算机工程, 2007, 33(3): 192-193, 196
[3]	Lafferty J D, McCallum A, Pereira F C N. Conditional random fields: probabilistic models for segmenting and labeling sequence data // Proceedings of the 18th International Conference on Machine Learning. Williamstown, 2001: 282-289
[4]	张合, 王晓东, 杨建宇, 等. 一种基于层叠 CRF 的古文断句与句读标记方法. 计算机应用研究, 2009, 26(9): 3326-3329
[5]	张开旭, 夏云庆, 宇航. 基于条件随机场的古汉语自动断句与标点方法. 清华大学学报: 自然科学版, 2009, 49(10): 1733-1736
[6]	Huang H H, Sun C T, Chen H H.Classical Chinese sentence segmentation // Proceedings of CIPS-SIGHAN Joint Conference on Chinese Language Processing. Beijing, 2010: 1-8
[7]	Wang B, Shi X, Tan Z, et al.A sentence segmentation method for ancient Chinese texts based on NNLM // Proceedings of CLSW. Singapore, 2016: 387-396
[8]	Graves A.Supervised sequence labelling with re-current neural networks. Studies in Computational Intelligence. Heidelberg: Springer, 2012: 22-29
[9]	Schuster M, Paliwal K K. Bidirectional recurrent neural networks. IEEE Transactions on Signal Proce-ssing, 1997, 45(11): 2673-2681
[10]	Cho K, Merrienboer B V, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation [EB/OL]. (2014-09-03) [2016-07-29].
[11]	Tieleman T, Hinton G.RMSProp: divide the gradient by a running average of its recent magnitude: report of COURSERA “Neural Networks for Machine Lear-ning”[R]. Toronto, 2012
[12]	Salimans T, Kingma D P. Weight normalization: a simple reparameterization to accelerate training of deep neural networks [EB/OL]. (2016-06-04) [2016-07-29].
[13]	Yao Y, Rosasco L, Caponnetto A. On early stopping in gradient descent learning. Constructive Approxi-mation, 2007, 26(2): 289-315
[14]	Hinton G E, Srivastava N, Krizhevsky A, et al. Improving neural networks by preventing coadapta-tion of feature detectors [EB/OL]. (2012-07-03) [2016- 07-29].
[15]	Pham V, Kermorvant C, Louradour J.Dropout improves recurrent neural networks for handwriting recognition[EB/OL]. (2014-03-10) [2016-07-29].
[16]	Moon T, Choi H, Lee H, et al. RNNDROP: a novel dropout for RNNS in ASR // IEEE Automatic Speech Recognition and Understanding Workshop. Scotts-dale, 2015: 65-70
[17]	Semeniuta S, Severyn A, Barth E. Recurrent dropout without memory loss [EB/OL]. (2016-03-16) [2016-07-29].
[18]	Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging [EB/OL]. (2015-08-09) [2016-07-29].
[19]	Chen X, Qiu X, Zhu C, et al.Long short-term memory neural networks for Chinese word segmen-tation // Proceedings of EMNLP. Lisbon, 2015: 1197-1206

数据集	规模	字表大小	内容	来源
训练集	2.37亿字	23905	“至善语料库”中的繁体古籍部分	至善语料库
开发集	1万字	1890	《阅微草堂笔记》(卷一)	汉籍
测试集1	32万字	6188	《宾退录》、《朝野佥载》、《南部新书》、《楚辞补注》和《中吴纪闻》	“是何年”网站
测试集2	36万字	5755	《阅微草堂笔记》(卷二至卷二十四)和《敬斋古今黈》	汉籍, “是何年”网站

数据集	规模	字表大小	内容	来源
训练集	2.37亿字	23905	“至善语料库”中的繁体古籍部分	至善语料库
开发集	1万字	1890	《阅微草堂笔记》(卷一)	汉籍
测试集1	32万字	6188	《宾退录》、《朝野佥载》、《南部新书》、《楚辞补注》和《中吴纪闻》	“是何年”网站
测试集2	36万字	5755	《阅微草堂笔记》(卷二至卷二十四)和《敬斋古今黈》	汉籍, “是何年”网站

模型	测试集1			测试集2
模型	P	R	F1	P	R	F1
CLM6	0.7822	0.5767	0.6639	0.7283	0.6364	0.6793
CLM6+0.65LP	0.7492	0.6727	0.7089	0.6861	0.7205	0.7029
CLM1+0.65LP	0.7212	0.7325	0.7268	0.6393	0.7691	0.6982
CRF	0.8163	0.6617	0.7309	0.7856	0.7100	0.7459
GRU-RNN	0.7545	0.6773	0.7138	0.7342	0.6329	0.6798
GRU-RNN+1.4ST (beam=50)	0.7851	0.7202	0.7513	0.7598	0.7187	0.7387
GRU-RNN+1.4ST+0.3LP (beam=50)	0.7606	0.7458	0.7531	0.7350	0.7533	0.7440
GRU-RNN+1.4ST+0.3LP (beam=200)	0.7629	0.7475	0.7551	0.7381	0.7564	0.7472

模型	测试集1			测试集2
模型	P	R	F1	P	R	F1
CLM6	0.7822	0.5767	0.6639	0.7283	0.6364	0.6793
CLM6+0.65LP	0.7492	0.6727	0.7089	0.6861	0.7205	0.7029
CLM1+0.65LP	0.7212	0.7325	0.7268	0.6393	0.7691	0.6982
CRF	0.8163	0.6617	0.7309	0.7856	0.7100	0.7459
GRU-RNN	0.7545	0.6773	0.7138	0.7342	0.6329	0.6798
GRU-RNN+1.4ST (beam=50)	0.7851	0.7202	0.7513	0.7598	0.7187	0.7387
GRU-RNN+1.4ST+0.3LP (beam=50)	0.7606	0.7458	0.7531	0.7350	0.7533	0.7440
GRU-RNN+1.4ST+0.3LP (beam=200)	0.7629	0.7475	0.7551	0.7381	0.7564	0.7472

A Sentence Segmentation Method for Ancient Chinese Texts Based on Recurrent Neural Network

一种基于循环神经网络的古文断句方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 19

Related Articles 1

Recommended Articles

Metrics