北京大学学报(自然科学版) ›› 2026, Vol. 62 ›› Issue (5): 977-985.DOI: 10.13209/j.0479-8023.2026.007

上一篇    下一篇

融合翻译难度的机器译文自动评价

唐铭, 李茂西†   

  1. 江西师范大学人工智能学院, 南昌 330022
  • 收稿日期:2025-07-31 修回日期:2025-09-11 出版日期:2026-09-20 发布日期:2026-09-20
  • 基金资助:
    国家自然科学基金(62366020, 61662031)和江西省教育厅研究生创新基金(YC2024-S271)资助

Automatic Evaluation of Machine Translation Incorporating Translation Difficulty

TANG Ming, LI Maoxi†   

  1. School of Artificial Intelligence, Jiangxi Normal University, Nanchang 330022
  • Received:2025-07-31 Revised:2025-09-11 Online:2026-09-20 Published:2026-09-20

摘要:

针对现有主流机器译文自动评价方法在评分时统一对待所有评价单元, 忽略源语言句子的翻译难度差异, 难以充分反映模型处理复杂语义结构或低频词时表现的问题, 提出一种融合翻译难度的机器译文自动评价方法TD_MT。该方法利用大语言模型分析代表性样本, 提取句法和词汇特征(如依存树深度和词频), 构建具备语言学可解释性的基于规则的翻译难度分类框架。依据源语句的预估难度等级, 对评价分数进行动态调整, 实现更精细的评估。在句子级别上, 与参与测评性能最优的机器译文自动评价方法XCOMET-Ensemble相比, 融合翻译难度的机器译文自动评价方法的相关性提高0.004; 在系统级别上, 相关性比XCOMET-Ensemble提高0.060。在WMT’23机器译文自动评价方法测评基准数据集上的实验结果表明, 与经典的机器译文自动评价方法和XCOMET-Ensemble相比, 融合翻译难度的机器译文自动评价方法提高了其与人类评价的相关性。

关键词: 机器翻译, 机器译文自动评价, 翻译难度, 大语言模型

Abstract:

Mainstream automatic evaluation metrics of machine translation typically treat all input sentences equally, overlooking the variation in translation difficulty posed by different source sentences. This often leads to inadequate reflection of system performance on structurally complex or lexically rare content. To address this limitation, we propose a novel evaluation approach TD_MT that integrates translation difficulty as a core factor in the scoring process. By analyzing representative samples using large language models and extracting syntactic and lexical features, such as dependency tree depth and word frequency, a linguistically interpretable, rule-based framework for translation difficulty classification is constructed. Evaluation scores are dynamically adjusted based on the estimated difficulty levels of source sentences, enabling a more nuanced assessment. At the sentence level, the proposed difficulty-aware machine translation evaluation metric exceeds the best-performing participating metric, XCOMET-Ensemble, by 0.004. At the system level, it exceeds XCOMET-Ensemble by 0.060. Experimental results on the WMT’23 benchmark dataset demonstrate that, compared with classical evaluation metrics and XCOMET-Ensemble, the proposed method achieves a significantly higher correlation with human judgments.

Key words: machine translation, automatic evaluation of machine translation, translation difficulty, large language model