Pruning and Fine-tuning Optimization Method of Convolutional Neural 
Network Based on Global Information

doi:10.13209/j.0479-8023.2021.053

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2021, Vol. 57 ›› Issue (4): 790-794.DOI: 10.13209/j.0479-8023.2021.053

Pruning and Fine-tuning Optimization Method of Convolutional Neural Network Based on Global Information

SUN Wenyu, CAO Jian^†, LI Pu, LIU Rui

School of Software and Microelectronics, Peking University, Beijing 102600

Received:2020-06-02 Revised:2020-07-08 Online:2021-07-20 Published:2021-07-20
Contact: CAO Jian, E-mail: caojian(at)ss.pku.edu.cn

基于全局信息的卷积神经网络模型剪枝微调优化方法

孙文宇, 曹健^†, 李普, 刘瑞

北京大学软件与微电子学院, 北京 102600

通讯作者: 曹健, E-mail: caojian(at)ss.pku.edu.cn
基金资助:
国家自然科学基金(U20A20204)资助

Abstract

Abstract:

In order to solve the problem that convolutional neural network is large and the accuracy loss of the model pruning method is relatively serious, a fine-tuning optimization method for model pruning is proposed. The global information of the original convolutional neural network model is introduced to the post-prune model to make it store the original model information which improves the accuracy of the model after pruning. Experimental results show that for the image classification tasks and target detection tasks, proposed fine-tuning optimization method can obtain greater compression ratio and smaller model accuracy loss.

Key words: convolutional neural network, model pruning and fine-tuning, global information, image classification, object detection

摘要：

为解决因卷积神经网络模型规模大, 模型剪枝方法引起的精度下降问题, 提出一种模型剪枝微调优化方法。该方法引入原卷积神经网络模型权重全局信息至剪枝后模型, 使原模型信息体现在剪枝后模型的权重上, 提升剪枝后模型的精度。在图像分类任务和目标检测任务中的实验结果表明, 所提出的微调优化方法可获得更大的压缩率和更小的模型精度损失。

关键词: 卷积神经网络, 模型剪枝微调, 全局信息, 图像分类, 目标检测

SUN Wenyu, CAO Jian, LI Pu, LIU Rui. Pruning and Fine-tuning Optimization Method of Convolutional Neural Network Based on Global Information[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(4): 790-794.

孙文宇, 曹健, 李普, 刘瑞. 基于全局信息的卷积神经网络模型剪枝微调优化方法[J]. 北京大学学报自然科学版, 2021, 57(4): 790-794.

[1]	XIE Hao, CAO Jian, LI Pu, ZHAO Xiongbo, ZHANG Xing. A Hardware Accelerator for SSD Object Detection Algorithm Based on FPGA [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(6): 1015-1022.
[2]	LI Fang, CAO Jian, LI Pu, XIE Hao, ZHAO Xiongbo, WANG Yuan, ZHANG Xing. Design and Implementation of Object Detection Acceleration Module Based on an ARM+FPGA Heterogeneous Platform [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(6): 1035-1041.
[3]	XU Pengtao, CAO Jian, SUN Wenyu, LI Pu, WANG Yuan, ZHANG Xing. Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(5): 801-807.
[4]	XU Pengtao, CAO Jian, CHEN Weiqian, LIU Shengrong, WANG Yuan, ZHANG Xing. Post Training Quantization Preprocessing Method of Convolutional Neural Network via Outlier Removal [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(5): 808-812.
[5]	LI Xingya, CHEN Yufeng, XU Jin’an, ZHANG Yujie. Distant Supervision for Relation Extraction with Gate Mechanism [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 39-44.
[6]	XU Haoliang, LI Yanqun, HE Yunqi, QIAN Longhua. Research on Chinese Nested Named Entity Relation Extraction [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(1): 8-14.
[7]	ZHANG Qinglin, DU Jiachen, XU Ruifeng. Sarcasm Detection Based on Adversarial Learning [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(1): 29-36.
[8]	LUO Fan, WANG Houfeng. Chinese Text Sentiment Classification by H-RNN-CNN [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2018, 54(3): 459-465.
[9]	Wenhao YING, Xinyan XIAO, Sujian LI, Yajuan LÜ, Zhifang SUI. Improving Query-Focused Summarization with CNN-Based Similarity [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2017, 53(2): 197-203.
[10]	CHAI Baohui, LI Peijun, ZHANG Ruijie, ZHAO Peng. Urban Expansion Extraction Using Landsat Series Data and DMSP/OLS Nighttime Light Data: A Case Study of Tianjin Area [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2016, 52(3): 475-485.
[11]	SHANG Kun,LI Peijun,CHENG Tao. Land Cover Classification of Hyperspectral Data Using Composite Kernel Support Vector Machines [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2011, 47(1): 109-114.
[12]	QIN Qiming,LU Rongjian. Satellite Image Classification Based on Fractal Dimension and Neural Networks [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2000, 36(6): 858-864.

Pruning and Fine-tuning Optimization Method of Convolutional Neural Network Based on Global Information

基于全局信息的卷积神经网络模型剪枝微调优化方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 12

Recommended Articles

Metrics