北京大学学报自然科学版 ›› 2023, Vol. 59 ›› Issue (1): 65-75.DOI: 10.13209/j.0479-8023.2022.065

上一篇    下一篇

基于预训练模型和混合神经网络的医疗实体关系抽取

赵丹丹, 张俊朋, 孟佳娜, 张志浩, 苏文   

  1. 大连民族大学计算机科学与工程学院, 大连 116600
  • 收稿日期:2022-05-09 修回日期:2022-07-26 出版日期:2023-01-20 发布日期:2023-01-20
  • 通讯作者: 孟佳娜, E-mail: mengjn(at)dlnu.edu.cn
  • 基金资助:
    国家自然科学基金(61876031)和国家科技创新 2030—“新一代人工智能”重大项目(2020AAA08000)资助

Medical Entity Relation Extraction Based on Pre-trained Model and Hybrid Neural Network

ZHAO Dandan, ZHANG Junpeng, MENG Jiana, ZHANG Zhihao, SU Wen   

  1. School of Computer Science and Engineering, Dalian Minzu University, Dalian 116600
  • Received:2022-05-09 Revised:2022-07-26 Online:2023-01-20 Published:2023-01-20
  • Contact: MENG Jiana, E-mail: mengjn(at)dlnu.edu.cn

摘要:

医疗文本具有实体密度高、句式冗长等特点, 简单的神经网络方法不能很好地捕获其语义特征, 因此提出一种基于预训练模型的混合神经网络方法。首先使用预训练模型获取动态词向量, 并提取实体标记特征; 然后通过双向长短期记忆网络获取医疗文本的上下文特征, 同时使用卷积神经网络获取文本的局部特征; 再使用注意力机制对序列特征进行加权, 获取文本全局语义特征; 最后将实体标记特征与全局语义特征融合, 并通过分类器得到抽取结果。在医疗领域数据集上的实体关系抽取实验结果表明, 新提出的混合神经网络模型的性能比主流模型均有提升, 说明这种多特征融合的方式可以提升实体关系抽取的效果。

关键词: 医疗文本, 关系抽取, 混合神经网络, 预训练模型

Abstract:

Medical text has high entity density and verbose sentence structure, which makes the simple neural network methods unable to capture its semantic features. Therefore, a hybrid neural network method based on pre-trained model is proposed. Firstly, a pre-trained model is used to obtain the dynamic word vector and the entity tagging features are extracted. Secondly, the contextual features of the medical text are obtained through a bidirectional long and short-term memory network. Simultaneously, the local features of the text are obtained using the convolutional neural network. Then the global semantic features of the text are obtained by weighting the sequence features through the attention mechanism. Finally, the entity tagging features are fused with the global semantic features and the extraction results are obtained through the classifier. The experimental results of entity relation extraction on the medical domain dataset show that the performance of the proposed hybrid neural network model is improved compared with the mainstream models, which indicates that this multi-feature fusion method can improve the effect of entity relation extraction. 

Key words: medical text, relation extraction, hybrid neural network, pre-trained model