Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2023, Vol. 59 ›› Issue (1): 30-38.DOI: 10.13209/j.0479-8023.2022.071

Previous Articles     Next Articles

A Short Text Matching Model Incorporating Contextual Semantic Differences

ZHANG Wenhui, WANG Meiling, HOU Zhirong†   

  1. ICBC Technology Co Ltd, Beijing 100029
  • Received:2022-05-13 Revised:2022-08-04 Online:2023-01-20 Published:2023-01-20
  • Contact: HOU Zhirong, E-mail: houzr(at)tech.icbc.com.cn

融合语境语义差异特征的短文本匹配模型

张文慧, 汪美玲, 侯志荣†   

  1. 工商银行金融科技研究院, 北京 100029
  • 通讯作者: 侯志荣, E-mail: houzr(at)tech.icbc.com.cn

PDF

Abstract:

Short text matching is often unable to accurately obtain the degree of semantic similarity between sentences when the semantic difference of the same wording and the semantic equivalence of the different wording. To solve this problem, the paper proposes a short text matching model which integrates contextual semantic differences. In this model, language models from the BERT series are utilized as a basic matching model, a novel Diff Transformer structure is implemented for extracting difference feature, and a gate mechanism is applied to integrate basic semantic representations and difference feature for a better matching effect. The model achieves the effect of advanced models on Chinese test datasets.

Key words: short text matching, difference feature, context semantic, Diff Transformer

摘要：

在字面相同语义不同和字面不同语义相同的情况下, 短文本匹配往往不能准确地得到语句间语义的相似程度。针对这一问题, 提出一种融合语境语义差异特征的短文本匹配模型。该模型以BERT系列的语言模型作为基础匹配模型, 采用一种新的Diff Transformer 结构作为差异特征提取器, 并以门控方式融合基础语义表示和差异特征表示来提升匹配效果。在中文测试数据集上的实验结果表明, 所提出的模型可以达到先进模型的效果。

关键词: 短文本匹配, 差异特征, 语境语义, Diff Transformer

Copyright © Acta Scientiarum Naturalium Universitatis Pekinensis, All Rights Reserved.
Powered by Beijing Magtech Co., Ltd.