Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2021, Vol. 57 ›› Issue (1): 31-37.DOI: 10.13209/j.0479-8023.2020.087

Previous Articles     Next Articles

Multi-Turn Conversation Rewriter Model Based on Masked-Pointer

YANG Shuangtao†, FU Bo, YU Chenchen, HU Changjian   

  1. AI Lab, Lenovo Research, Beijing 100085
  • Received:2020-06-16 Revised:2020-08-10 Online:2021-01-20 Published:2021-01-20
  • Contact: YANG Shuangtao，E-mail: 460130107(at)qq.com

基于Masked-Pointer的多轮对话重写模型

杨双涛†, 符博, 于晨晨, 胡长建   

  1. 联想研究院人工智能实验室, 北京 100085
  • 通讯作者: 杨双涛，E-mail: 460130107(at)qq.com

PDF

Abstract:

To solve the problem of Non-Sentential Utterances in multi-turn conversations, Masked Rewriter Model is proposed based on the Masked Language Model, and the rewriting performance is significantly improved compared with the Seq2Seq-based rewriting model. Considering the NSUs rewriting task characteristics, Masked-Pointer Rewriter Model is proposed based on the Masked Language Model and Pointer Network, which achieves better rewriting results than the Masked Rewriter Model by using the Pointer Network to enhance the model’s attention to historical information.

Key words: human-computer interaction, pre-trainied language model, pointer network, conversation rewrite

摘要：

针对多轮会话中的Non-Sentential Utterances (NSUs)问题, 结合当前在自然语言处理领域广泛使用的预训练语言模型, 将Masked Language Model用于多轮会话NSUs的重写任务, 提出Masked Rewriter Model。与基于Seq2Seq的重写模型相比, 重写效果提升明显。根据NSUs重写任务特点, 将Masked Language Model与Pointer Network相结合, 提出基于Masked-Pointer Rewriter Model的多轮会话重写模型, 利用指针网络, 提升重写模型对上文信息的关注程度, 在BERT Masked Rewriter模型的基础上进一步提升重写效果。

关键词: 人机交互, 预训练语言模型, 指针网络, 会话重写

Copyright © Acta Scientiarum Naturalium Universitatis Pekinensis, All Rights Reserved.
Powered by Beijing Magtech Co., Ltd.