An Abstractive Summarization Method Based on Encoder-Sharing and Gated Network

doi:10.13209/j.0479-8023.2019.100

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2020, Vol. 56 ›› Issue (1): 61-67.DOI: 10.13209/j.0479-8023.2019.100

Previous Articles Next Articles

An Abstractive Summarization Method Based on Encoder-Sharing and Gated Network

TIAN Keke^1,2, ZHOU Ruiying^1,2, DONG Haoye^1,2, YIN Jian^1,2,†

1. School of Data and Computer Science, Sun Yat-Sen University, Guangzhou 510006 2. Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou 510006

Received:2019-05-22 Revised:2019-09-25 Online:2020-01-20 Published:2020-01-20
Contact: YIN Jian, E-mail: issjyin(at)mail.sysu.edu.cn

基于编码器共享和门控网络的生成式文本摘要方法

田珂珂^1,2, 周瑞莹^1,2, 董浩业^1,2, 印鉴^1,2,†

1. 中山大学数据科学与计算机学院, 广州 510006 2. 广东省大数据分析与处理重点实验室, 广州 510006

通讯作者: 印鉴, E-mail: issjyin(at)mail.sysu.edu.cn
基金资助:
广东省科技计划项目(2015A030401057, 2016B030307002, 2017B030308007)资助

Abstract

Abstract:

This paper proposed an abstractive summarization method based on self-attention based Transformer model, which regarded encoder as part of decoder, and used gated network to control the information flow from encoder to decoder. Compared with the existing methods, proposed method improves the training and inference speed of text summarization task, and improves the accuracy and fluency of generating summary. Experiments on English summarization dataset Gigaword and DUC2004 demonstrate that proposed model outperforms the baseline models on both the quality of summarization and time efficiency.

Key words: abstractive, summarization, self-attention, encoder-sharing, gated network

摘要：

结合基于自注意力机制的Transformer模型, 提出一种基于编码器共享和门控网络的文本摘要方法。该方法将编码器作为解码器的一部分, 使解码器的部分模块共享编码器的参数, 同时使用门控网络筛选输入序列中的关键信息。相对已有方法, 所提方法提升了文本摘要任务的训练和推理速度, 同时提升了生成摘要的准确性和流畅性。在英文数据集Gigaword和DUC2004上的实验表明, 所提方法在时间效率和生成摘要质量上, 明显优于已有模型。

关键词: 生成式, 文本摘要, 自注意力机制, 编码器共享, 门控网络

TIAN Keke, ZHOU Ruiying, DONG Haoye, YIN Jian. An Abstractive Summarization Method Based on Encoder-Sharing and Gated Network[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 61-67.

田珂珂, 周瑞莹, 董浩业, 印鉴. 基于编码器共享和门控网络的生成式文本摘要方法[J]. 北京大学学报自然科学版, 2020, 56(1): 61-67.

[1]	WANG Haonan, GAO Yang, FENG Junlan, HU Min, WANG Huixin, BAI Yu. Abstractive Summarization Based on Fine-Grained Interpretable Matrix [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 23-30.
[2]	WU Shixin, HUANG Degen, LI Jiuyi. Abstractive Text Summarization Based on Semantic Alignment Network [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 1-6.
[3]	LIN Xinyi, YAN Rui, ZHAO Dongyan. A Hybrid Optimization Framework Fusing Word- and Sentence-Level Information for Extractive Summarization [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2018, 54(2): 229-235.
[4]	Wenhao YING, Xinyan XIAO, Sujian LI, Yajuan LÜ, Zhifang SUI. Improving Query-Focused Summarization with CNN-Based Similarity [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2017, 53(2): 197-203.
[5]	LIU Jiangming,XU Jin’an,ZHANG Yujie. Summarization Based on Hidden Topic Markov Model with Multi-features [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2014, 50(1): 187-193.
[6]	HUANG Xiaojiang,WAN Xiaojun,XIAO Jianguo. Comparative News Summarization Using Co-ranking Graph Model [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2013, 49(1): 31-38.
[7]	TONG Yijian,TANG Huifeng. Topic Partition for Automatic Summarization [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2013, 49(1): 39-44.

An Abstractive Summarization Method Based on Encoder-Sharing and Gated Network

基于编码器共享和门控网络的生成式文本摘要方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 7

Recommended Articles

Metrics