Discourse-Level Text Generation Method Based on Topical Constraint

doi:10.13209/j.0479-8023.2019.103

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2020, Vol. 56 ›› Issue (1): 9-15.DOI: 10.13209/j.0479-8023.2019.103

Previous Articles Next Articles

Discourse-Level Text Generation Method Based on Topical Constraint

HUANG Yan^1,2, SUN Haili¹, XU Ke^1,3, YU Xiaoyang¹, WANG Tongyang^1,†, ZHANG Xinfang¹, LU Songfeng^1,2

1. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074 2. Shenzhen Huazhong University of Science and Technology Research Institute, Shenzhen 518063 3. School of Computer Science, South-Central University for Nationalities, Wuhan 430074

Received:2019-05-22 Revised:2019-09-23 Online:2020-01-20 Published:2020-01-20
Contact: WANG Tongyang, E-mail: platanus(at)hust.edu.cn

基于主题约束的篇章级文本生成方法

黄炎^1,2, 孙海丽¹, 徐科^1,3, 余晓阳¹, 王同洋^1,†, 张新访¹, 路松峰^1,2

1. 华中科技大学计算机科学与技术学院, 武汉 430074 2. 深圳华中科技大学研究院, 深圳 518063
3. 中南民族大学计算机科学学院, 武汉 430074

通讯作者: 王同洋, E-mail: platanus(at)hust.edu.cn
基金资助:
深圳市科技计划基础研究项目(JCYJ20180306124612893, JCYJ20170818160208570, JCYJ20170307160458368)资助

Abstract

Abstract:

To solve the topic missing problem of text generated by computers, this paper proposed a new discourse-level text generation method based on topical constraint. Providing a short topic description, the approach extracted several topic words from the text, then extended and clustered the keywords to form topical planning which were used to restrain the generation of each paragraphs. The model improved the attention based recurrent neural network form three aspects including topic distribution, attention scoring function and topic coverage generation. In experiments, the proposed method was compared with benchmark models such as Char-RNN, SC-LSTM and MTA-LSTM on three real datasets, three improvement strategies were verified and analysed independently. Experimental results show that proposed model is more efficient than benchmark models on human and BLEU metrics, and the generated text can catch the topic more effectively.

Key words: automatic text generation, topical constraint, RNN, LSTM, attention mechanism

摘要：

针对计算机自动生成的文本缺乏主题思想这一问题, 提出一种基于主题约束的篇章级文本自动生成方法。该方法围绕用户输入的主题描述语句提取若干主题词; 然后对主题词进行扩展和主题聚类, 形成文章主题规划; 最后利用每个聚类中的关键词信息约束每个段落的文本生成。该模型从文本主题分布、注意力评分方法和主题覆盖生成3个方面对现有基于注意力机制的循环神经网络文本生成模型进行了改进。在3个真实数据集上分别与Char-RNN, SC-LSTM和MTA-LSTM基准模型进行对比, 并对3个方面的改进进行独立验证。实验结果表明, 所提方法在人工评判和BLEU自动评测上均优于基准模型, 生成的文本能更好地贴合主题。

关键词: 文本自动生成, 主题约束, 循环神经网络(RNN), 长短时记忆网络(LSTM), 注意力机制

HUANG Yan, SUN Haili, XU Ke, YU Xiaoyang, WANG Tongyang, ZHANG Xinfang, LU Songfeng. Discourse-Level Text Generation Method Based on Topical Constraint[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 9-15.

黄炎, 孙海丽, 徐科, 余晓阳, 王同洋, 张新访, 路松峰. 基于主题约束的篇章级文本生成方法[J]. 北京大学学报自然科学版, 2020, 56(1): 9-15.

[1]	LI Chao, HOU Xia, QIAO Xiuming. A Low-Resource Named Entity Recognition Method for Cultural Heritage Field Incorporating Knowledge Fusion [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 13-22.
[2]	WANG Rui, LIANG Jianguo, HUA Rong. Radiology Report Generation Method Based on Multi-scale Feature Parsing [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 71-78.
[3]	DING Na, LIU Peng, SHAO Huipeng, WANG Xuekui. Bi-Attention Text-Keyword Matching for Law Recommendation [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 79-88.
[4]	LI Dai, WANG Tianmu, ZHANG Si, QIN Yue, XIE Fugui, LIU Xinjun, NIE Zhenguo, HUANG Hongshi. Intelligent Diagnosis on Anterior Cruciate Ligament Deficiency Based on Plantar Pressure and ConvLSTM Neural Network [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 109-117.
[5]	TIAN Yuan, PANG Xiao, ZHAO Wenyi, CHANG Xiaoyin, CHENG Chuyun, ZOU Pei, CAO Xiaocheng. A Transformer-Based Model for Short-Term Landslide Displacement Prediction [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(2): 197-210.
[6]	GAO Xiaoqian, ZHOU Xiabing, ZHANG Min. A Multi-information Perception Based Method for Question Answering in Multi-party Conversation [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(1): 21-29.
[7]	SU Fenglong, SUN Chengzhe, JING Ning. A Context-Fusion Method for Entity Extraction Based on Residual Gated Convolution Neural Network [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(1): 69-76.
[8]	WANG Yinmiao, HAN Zhimin. Knowledge Bases Completion Based on Multi-hop Paths [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(1): 7-12.
[9]	ZHOU Pei, HUANG Yingjie, HU Bingyi, WEI Jun. Spring Predictability Barrier Phenomenon in ENSO Prediction Model Based on LSTM Deep Learning Algorithm [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(6): 1071-1078.
[10]	LI Zequn, WEI Jun. Prediction of PM_2.5 Daily Concentration of Guangzhou Based on Neural Network Algorithms [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(4): 645-652.
[11]	WAN Zhang, ZHANG Yujie, LIU Mingtong, XU Jin’an, CHEN Yufeng. Object Space Relation Mechanism Fused Image Caption Method [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 75-82.
[12]	LIANG Wanying, ZHU Jia, WU Zhijie, YAN Zhiwen, TANG Yong, HUANG Jin, YU Weihao. Syntax-based Code Generation Model with Selective Local Attention and Pre-order Information LSTM Decoder [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 45-52.
[13]	HUANG Depeng, LI Zhenghua, GONG Chen, ZHANG Min. Neural Network Coupled Model for Conversion and Exploitation of Heterogeneous Lexical Annotations [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 97-104.
[14]	WU Ruiying, KONG Fang. Event Coreference Resolution with Document Representation [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 82-88.
[15]	JIA Yunlong, HAN Donghong, LIN Haiyuan, WANG Guoren, XIA Li. Consumption Intent Recognition Algorithms for Weibo Users [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 68-74.

Discourse-Level Text Generation Method Based on Topical Constraint

基于主题约束的篇章级文本生成方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics