A Transformer-Based Model for Short-Term Landslide Displacement Prediction

doi:10.13209/j.0479-8023.2022.106

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2023, Vol. 59 ›› Issue (2): 197-210.DOI: 10.13209/j.0479-8023.2022.106

Previous Articles Next Articles

A Transformer-Based Model for Short-Term Landslide Displacement Prediction

TIAN Yuan^1,2, PANG Xiao^1,2, ZHAO Wenyi^3,†, CHANG Xiaoyin^1,2, CHENG Chuyun^1,2, ZOU Pei^4,5, CAO Xiaocheng^1,2

1. Institute of Remote Sensing and Geographical Information Systems, Peking University, Beijing 100871 2. Beijing Key Laboratory of Spatial Information Integration and Its Applications, Beijing 100871 3. China Institute of Geo- Environment Monitoring, Beijing 100081 4. School of Computer Science, Peking University, Beijing 100871 5. Key Lab of Computational Linguistics (MOE), Beijing 100871

Received:2022-03-11 Revised:2022-04-21 Online:2023-03-20 Published:2023-03-20
Contact: ZHAO Wenyi, E-mail: 395447712(at)qq.com

基于Transformer的滑坡短期位移预测模型

田原^1,2, 庞骁^1,2, 赵文祎^3,†, 常啸寅^1,2, 程楚云^1,2, 邹佩^4,5, 曹晓澄^1,2

1. 北京大学遥感与地理信息系统研究所, 北京 100871 2. 空间信息集成与 3S 工程应用北京市重点实验室, 北京 100871 3. 中国地质环境监测院, 100081 4. 北京大学计算机学院, 100871 5. 计算语言学教育部重点实验室, 北京 100871

通讯作者: 赵文祎, E-mail: 395447712(at)qq.com
基金资助:
中国地质调查局地质调查项目(DD20211364)和国家重点研发计划(2021YFC3000504-02)资助

Abstract

Abstract:

A Transformer-based short-term landslide displacement prediction model is proposed by combining temporal convolutional network (TCN) with a Transformer decoder. This model takes the preprocessed displacement and rainfall sequences as input and outputs the displacement predictions for the next three days in a time-series autoregressive manner. The experimental results show that the model achieves higher prediction accuracy than support vector machine (SVM) and long short-term memory (LSTM), and performs particularly well during predicting rapid deformation periods. At the same time, through the analysis of the attention mechanism of the model, it is found that the model focuses on the peak of displacements and heavy rainfalls, indicating that the model is reasonably reliable.

Key words: slope engineering, landslide displacement, short-term prediction, Transformer, attention mechanism

摘要：

通过将时序卷积网络(TCN)与Transformer解码器进行组合, 提出一种基于Transformer的滑坡短期位移预测模型。将预处理过的位移与降雨序列作为模型的输入, 以时序自回归方式输出未来3日的位移预测结果。实验结果表明, 与支持向量机(SVM)和长短期记忆(LSTM)等传统模型相比, 该模型精度较高, 在快速变形期的预测优势尤为突出。对模型注意力机制的分析结果表明, 模型关注的重点在位移峰值和大降雨附近, 具有较高的可信度。

关键词: 边坡工程, 滑坡位移, 短期预测, Transformer, 注意力机制

TIAN Yuan, PANG Xiao, ZHAO Wenyi, CHANG Xiaoyin, CHENG Chuyun, ZOU Pei, CAO Xiaocheng. A Transformer-Based Model for Short-Term Landslide Displacement Prediction[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(2): 197-210.

田原, 庞骁, 赵文祎, 常啸寅, 程楚云, 邹佩, 曹晓澄. 基于Transformer的滑坡短期位移预测模型[J]. 北京大学学报自然科学版, 2023, 59(2): 197-210.

[1]	LI Chao, HOU Xia, QIAO Xiuming. A Low-Resource Named Entity Recognition Method for Cultural Heritage Field Incorporating Knowledge Fusion [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 13-22.
[2]	ZHANG Naizhou, CAO Wei. A Context-Aware Query Suggestion Method Based on Multi-source Data Augmentation through Cross-Attention [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 34-42.
[3]	WANG Rui, LIANG Jianguo, HUA Rong. Radiology Report Generation Method Based on Multi-scale Feature Parsing [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 71-78.
[4]	DING Na, LIU Peng, SHAO Huipeng, WANG Xuekui. Bi-Attention Text-Keyword Matching for Law Recommendation [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2024, 60(1): 79-88.
[5]	ZHOU Bohan, CAO Jian, WANG Yuan. A Transformer-based Syntax Tree Decoder for Handwritten Mathematical Expression Recognition [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(6): 909-914.
[6]	GAO Xiaoqian, ZHOU Xiabing, ZHANG Min. A Multi-information Perception Based Method for Question Answering in Multi-party Conversation [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(1): 21-29.
[7]	ZHANG Wenhui, WANG Meiling, HOU Zhirong. A Short Text Matching Model Incorporating Contextual Semantic Differences [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(1): 30-38.
[8]	SU Fenglong, SUN Chengzhe, JING Ning. A Context-Fusion Method for Entity Extraction Based on Residual Gated Convolution Neural Network [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(1): 69-76.
[9]	WANG Qian, LI Maoxi, WU Shuixiu, WANG Mingwen. Neural Machine Translation Based on XLM-R Cross-lingual Pre-training Language Model [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(1): 29-36.
[10]	WANG Yinmiao, HAN Zhimin. Knowledge Bases Completion Based on Multi-hop Paths [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(1): 7-12.
[11]	WAN Zhang, ZHANG Yujie, LIU Mingtong, XU Jin’an, CHEN Yufeng. Object Space Relation Mechanism Fused Image Caption Method [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 75-82.
[12]	DUAN Jianyong, YUAN Yang, WANG Hao. Chinese Spelling Correction Method Based on Transformer Local Information and Syntax Enhancement Architecture [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 61-67.
[13]	ZHU Qinpei, MIAO Qingliang. An Antinoise Response Generation for Open Domain Dialogue System [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1): 38-44.
[14]	WU Ruiying, KONG Fang. Event Coreference Resolution with Document Representation [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 82-88.
[15]	JIA Yunlong, HAN Donghong, LIN Haiyuan, WANG Guoren, XIA Li. Consumption Intent Recognition Algorithms for Weibo Users [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 68-74.

A Transformer-Based Model for Short-Term Landslide Displacement Prediction

基于Transformer的滑坡短期位移预测模型

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics