A Transformer-based Syntax Tree Decoder for Handwritten Mathematical Expression Recognition

doi:10.13209/j.0479-8023.2023.085

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2023, Vol. 59 ›› Issue (6): 909-914.DOI: 10.13209/j.0479-8023.2023.085

Previous Articles Next Articles

A Transformer-based Syntax Tree Decoder for Handwritten Mathematical Expression Recognition

ZHOU Bohan, CAO Jian^†, WANG Yuan

School of Software and Microelectronics, Peking University, Beijing 102600

Received:2022-11-16 Revised:2023-01-10 Online:2023-11-20 Published:2023-11-20
Contact: CAO Jian, E-mail: caojian(at)ss.pku.edu.cn

基于Transformer模型的手写数学公式语法树解码器

周伯瀚, 曹健^†, 王源

北京大学软件与微电子学院, 北京 102600

通讯作者: 曹健, E-mail: caojian(at)ss.pku.edu.cn

Abstract

Abstract:

Most of the existing tree-structured decoding methods of handwritten mathematical expression recognition are based on the recurrent neural networks, which have low training efficiency and complicated training process. In order to prove this problem, the authors propose a handwritten mathematical expression recognition model based on Transformer structure, which can decode the syntax tree of expressions directly. Experimental results show that the proposed tree-structured decoding method achieves better performance than the string decoding methods base on Transformer on several datasets of handwritten formula recognition tasks, and show the potential to surpass recurrent neural network tree decoding methods.

Key words: handwritten mathematical expression recognition, Transformer, tree decoder, document comprehension

摘要：

目前对数学公式进行树结构解码的方法大多基于循环神经网络的结构, 训练效率低, 训练过程复杂, 基于此问题, 提出一种基于Transformer结构的手写数学公式识别模型, 可以直接对公式的语法树进行解码。在手写公式识别任务多个数据集上的实验结果表明, 所提出的Transformer树解码方法都取得超越Transformer序列解码方法的性能, 并展现出超越循环神经网络树解码方法的潜力。

关键词: 手写数学公式识别, Transformer, 树解码器, 图表理解

ZHOU Bohan, CAO Jian, WANG Yuan. A Transformer-based Syntax Tree Decoder for Handwritten Mathematical Expression Recognition[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(6): 909-914.

周伯瀚, 曹健, 王源. 基于Transformer模型的手写数学公式语法树解码器[J]. 北京大学学报自然科学版, 2023, 59(6): 909-914.

[1]	WANG Rui, LIANG Jianguo, HUA Rong. Radiology Report Generation Method Based on Multi-scale Feature Parsing [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 71-78.
[2]	ZHANG Naizhou, CAO Wei. A Context-Aware Query Suggestion Method Based on Multi-source Data Augmentation through Cross-Attention [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 34-42.
[3]	TIAN Yuan, PANG Xiao, ZHAO Wenyi, CHANG Xiaoyin, CHENG Chuyun, ZOU Pei, CAO Xiaocheng. A Transformer-Based Model for Short-Term Landslide Displacement Prediction [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(2): 197-210.
[4]	ZHANG Wenhui, WANG Meiling, HOU Zhirong. A Short Text Matching Model Incorporating Contextual Semantic Differences [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(1): 30-38.
[5]	WANG Qian, LI Maoxi, WU Shuixiu, WANG Mingwen. Neural Machine Translation Based on XLM-R Cross-lingual Pre-training Language Model [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(1): 29-36.
[6]	WAN Zhang, ZHANG Yujie, LIU Mingtong, XU Jin’an, CHEN Yufeng. Object Space Relation Mechanism Fused Image Caption Method [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 75-82.
[7]	DUAN Jianyong, YUAN Yang, WANG Hao. Chinese Spelling Correction Method Based on Transformer Local Information and Syntax Enhancement Architecture [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 61-67.
[8]	ZHU Qinpei, MIAO Qingliang. An Antinoise Response Generation for Open Domain Dialogue System [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 38-44.
[9]	LI Xiaonan, LIU Guoqiang, ZHANG Laifu, WANG Zhipeng, DING Guangxin, ZHAO Shilong, XIA Hui. Research on Transformer Oil Aging by Nuclear Magnetic Resonance Spectroscopy of Relaxation Time [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(5): 865-870.
[10]	CHANG Bingguo,YAN Lei,MAO Jietai,LIU Junhua. Observator with Fuzzy Neural Networks for Monitoring the Operation-state of Transformer [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2003, 39(1): 35-39.

A Transformer-based Syntax Tree Decoder for Handwritten Mathematical Expression Recognition

基于Transformer模型的手写数学公式语法树解码器

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 10

Recommended Articles

Metrics