An Antinoise Response Generation for Open Domain Dialogue System

doi:10.13209/j.0479-8023.2020.089

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2021, Vol. 57 ›› Issue (1): 38-44.DOI: 10.13209/j.0479-8023.2020.089

Previous Articles Next Articles

An Antinoise Response Generation for Open Domain Dialogue System

ZHU Qinpei^†, MIAO Qingliang

AI Speech Co., Ltd., Suzhou 215000

Received:2020-06-04 Revised:2020-08-10 Online:2021-01-20 Published:2021-01-20
Contact: ZHU Qinpei, E-mail: ross.zhu(at)aispeech.com

开放域对话系统的抗噪回复生成模型

朱钦佩^†, 缪庆亮

苏州思必驰信息科技有限公司, 苏州 215000

通讯作者: 朱钦佩, E-mail: ross.zhu(at)aispeech.com

Abstract

Abstract:

In order to reduce the noise interference on the response generation model, this paper proposes an antinoise model based on encoder-decoder architecture. Firstly, simulation noisy characters are added to the input utterances. Then noisy character recognition is trained at the encoder output layer, thus improving the ability of extracting noise features. Finally, pre-trained language model is fused at the encoder output layer to expand the coverage of noise. An antinoise test set is presented for verifying the model’s antinoise effect, which is the first Chinese single-turn open domain dialog system corpus with real noise. Experiments show that the proposed model’s results of automatic evaluation and manual evaluation on the antinoise test set are better than the baseline models.

Key words: natural language generation, pre-training language models, BERT, Transformer model

摘要：

为缓解输入语句中噪声对回复生成模型的干扰, 提出一个基于编码–解码框架的抗噪模型。首先，在训练集输入序列中随机加入模拟噪声字符; 然后，在编码端输出层训练噪声字符识别, 提升模型对噪声特征的提取能力; 最后, 在编码端输出层融合预训练语言模型, 扩大模型对噪声的覆盖面。为验证该模型的抗噪效果, 构建首个带真实噪声的单轮开放域闲聊系统抗噪能力测试集。在该测试集上的实验结果表明, 所提出的抗噪模型自动评测和人工评测结果均优于基准模型。

关键词: 自然语言生成, 预训练语言模型, BERT, Transformer模型

ZHU Qinpei, MIAO Qingliang. An Antinoise Response Generation for Open Domain Dialogue System[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 38-44.

朱钦佩, 缪庆亮. 开放域对话系统的抗噪回复生成模型[J]. 北京大学学报自然科学版, 2021, 57(1): 38-44.

Add to citation manager EndNote|Ris|BibTeX

URL: https://xbna.pku.edu.cn/EN/10.13209/j.0479-8023.2020.089

https://xbna.pku.edu.cn/EN/Y2021/V57/I1/38

[1]	ZHANG Naizhou, CAO Wei. A Context-Aware Query Suggestion Method Based on Multi-source Data Augmentation through Cross-Attention [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 34-42.
[2]	KONG Xiangfu, DONG Bo, XU Ke, TAO Yongliang. Text Classification Model for Livelihood Issues Based on BERT: A Study Based on Hotline Compliant Data of Zhejiang Province [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(3): 456-466.
[3]	YU Yuanfang, ZHANG Yong, ZUO Haoyang, ZHANG Lianfa, WANG Tingting. Multi-turn Event Argument Extraction Based on Role Information Guidance [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(1): 83-91.
[4]	YOU Xindong, GE Haojie, HAN Junmei, LI Yuxian, LÜ Xueqiang. Recognition of Complex Entities in Weapons and Equipment Field [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(3): 391-404.
[5]	DUAN Jianyong, YUAN Yang, WANG Hao. Chinese Spelling Correction Method Based on Transformer Local Information and Syntax Enhancement Architecture [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 61-67.
[6]	ZHAI Weixin, CHEN Bo, TONG Xiaochong, CHENG Chengqi. Research on Continuity of Multi-Scale Space-Filling Curves [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2018, 54(2): 331-335.
[7]	WEN Guochun. Discontinuous Riemann-Hilbert Problem for Analytic Functions in Multiply Connected Domains [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2013, 49(6): 937-944.
[8]	JIANG Mei,ZHANG Xing,WANG Xin’an,LIU Shan,XU Feng,WANG Bo,ZONG Hongqiang,SHEN Jinpeng. A High Performance Double-Balanced Gilbert Mixer with Active Loads for 2.4-GHz ISM Band Application [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2012, 48(4): 538-544.
[9]	WEI Jianping. Wave Propagationin Elastic Square Columns Ⅳ. Effect of Boundary Condition [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2009, 45(5): 743-747.
[10]	JIN Jian,WANG Weiwen,MENG Qingxuan,LIU Mei,SHAO Feng . Effects of Pubertal and Adult Social Isolation on the Latent Inhibition of Rats [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2009, 45(2): 350-354.
[11]	ZHANG Bei,TAN Benkui. Observational Analysis of Long Life-Cycle Baroclinic Wave Packets in Southern Hemisphere Summer Seasons [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2006, 42(2): 215-219.
[12]	WEN Guochun. The Riemann-Hilbert Problem for Linear Degenerate Elliptic Complex Equations of First Order [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2005, 41(1): 47-54.
[13]	WEN Guochun,TIAN Maoying. Generalizations of Keldych-Sedov Formula and Their Applications [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 1999, 35(1): 47-55.

An Antinoise Response Generation for Open Domain Dialogue System

开放域对话系统的抗噪回复生成模型

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 13

Recommended Articles

Metrics