Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2021, Vol. 57 ›› Issue (1): 16-22.DOI: 10.13209/j.0479-8023.2020.088

Previous Articles     Next Articles

Robustness of Chinese Machine Reading Comprehension

LI Yeqiu1, TANG Hongxuan1, QIAN Jin1, ZOU Bowei1,2, HONG Yu1,†   

  1. 1. School of Computer Science and Technology, Soochow University, Suzhou 215000 2. Institute for Infocomm Research, Singapore 138632
  • Received:2020-06-08 Revised:2020-08-14 Online:2021-01-20 Published:2021-01-20
  • Contact: HONG Yu, E-mail: tianxianer(at)gmail.com

中文机器阅读理解的鲁棒性研究

李烨秋1, 唐竑轩1, 钱锦1, 邹博伟1,2, 洪宇1,†   

  1. 1. 苏州大学计算机科学与技术学院, 苏州 215000 2. 新加坡资讯通信研究院, 新加坡138632
  • 通讯作者: 洪宇, E-mail: tianxianer(at)gmail.com
  • 基金资助:
    国家自然科学基金(61703293, 61672368, 61672367)和江苏高校优势学科建设工程项目资助

PDF

Abstract:

In order to better evaluate the robustness of Machine Reading Comprehension (MRC) models, this paper builds three test sets from Dureader by automatically extracting and manually annotating, consisting of oversensitivity, over-stability, and generalization. In addition, this paper proposes a multi-task learning framework with answer extraction task and masked position prediction task. Experimental results demonstrate that proposed method gains significant robustness improvements and show the effectiveness of the three test sets on evaluating the robustness of MRC models.

Key words: machine reading comprehension, robustness, Chinese corpus

摘要：

为了更好地评价阅读理解模型的鲁棒性, 基于Dureader数据集, 通过自动抽取和人工标注的方法, 对过敏感、过稳定和泛化3个问题分别构建测试数据集。还提出基于答案抽取和掩码位置预测的多任务学习方法。实验结果表明, 所提方法能显著地提高阅读理解模型的鲁棒性, 所构建的测试集能够对模型的鲁棒性进行有效评估。

关键词: 机器阅读理解, 鲁棒性, 中文语料库

Copyright © Acta Scientiarum Naturalium Universitatis Pekinensis, All Rights Reserved.
Powered by Beijing Magtech Co., Ltd.