Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks

doi:10.13209/j.0479-8023.2022.081

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2022, Vol. 58 ›› Issue (5): 801-807.DOI: 10.13209/j.0479-8023.2022.081

Previous Articles Next Articles

Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks

XU Pengtao, CAO Jian^†, SUN Wenyu, LI Pu, WANG Yuan, ZHANG Xing^†

School of Software and Microelectronics, Peking University, Beijing 102600

Received:2021-09-30 Revised:2021-12-04 Online:2022-09-20 Published:2022-09-20
Contact: CAO Jian, E-mail: caojian@ss.pku.edu.cn, ZHANG Xing, E-mail: zhx@pku.edu.cn

基于可融合残差卷积块的深度神经网络模型层剪枝方法

徐鹏涛, 曹健^†, 孙文宇, 李普, 王源, 张兴^†

北京大学软件与微电子学院, 北京 102600

通讯作者: 曹健, E-mail: caojian@ss.pku.edu.cn, 张兴, E-mail: zhx@pku.edu.cn
基金资助:
国家重点研发计划(2018YFE0203801)资助

Abstract

Abstract:

Aiming at the problems of long inference time and poor effect of the compression model obtained by the current mainstream pruning methods, an easy-to-use and excellent layer pruning method is proposed. The original convolution layers in the model are transformed into fusible residual convolutional blocks, and then layer pruning is realized by sparse training, therefore a layer pruning method with engineering ease is obtained, which has the advantages of short inference time and good pruning effect. The experimental results show that the proposed layer pruning method can achieve a very high compression rate with less accuracy loss in image classification tasks and object detection tasks, and the compression performance is better than the advanced convolutional kernel pruning methods.

Key words: convolutional neural network, layer pruning, fusible residual convolutional block, sparse training; image classification

摘要：

针对当前主流的剪枝方法所获得的压缩模型推理时间较长和效果较差的问题, 提出一种易用且性能优异的层剪枝方法。该方法将原始卷积层转化为可融合残差卷积块, 然后通过稀疏化训练的方法实现层剪枝, 得到一种具有工程易用性的层剪枝方法, 兼具推理时间短和剪枝效果好的优点。实验结果表明, 在图像分类任务和目标检测任务中, 该方法可使模型在精度损失较小的情况下获得极高的压缩率, 优于先进的卷积核剪枝方法。

关键词: 卷积神经网络, 层剪枝, 可融合残差卷积块, 稀疏化训练, 图像分类

XU Pengtao, CAO Jian, SUN Wenyu, LI Pu, WANG Yuan, ZHANG Xing. Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(5): 801-807.

徐鹏涛, 曹健, 孙文宇, 李普, 王源, 张兴. 基于可融合残差卷积块的深度神经网络模型层剪枝方法[J]. 北京大学学报自然科学版, 2022, 58(5): 801-807.

[1]	LIU Xiangcheng, CAO Jian, YAO Hongyi, XU Pengtao, ZHANG Yuan, WANG Yuan. AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(5): 764-772.
[2]	ZHOU Xiangyu, MAO Shanjun, LI Mei. Bearing Fault Diagnosis Method Based on Down-Sampling in Frequency Domain and CNN [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(2): 251-260.
[3]	XIE Hao, CAO Jian, LI Pu, ZHAO Xiongbo, ZHANG Xing. A Hardware Accelerator for SSD Object Detection Algorithm Based on FPGA [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(6): 1015-1022.
[4]	XU Pengtao, CAO Jian, CHEN Weiqian, LIU Shengrong, WANG Yuan, ZHANG Xing. Post Training Quantization Preprocessing Method of Convolutional Neural Network via Outlier Removal [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(5): 808-812.
[5]	SUN Wenyu, CAO Jian, LI Pu, LIU Rui. Pruning and Fine-tuning Optimization Method of Convolutional Neural Network Based on Global Information [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(4): 790-794.
[6]	LI Xingya, CHEN Yufeng, XU Jin’an, ZHANG Yujie. Distant Supervision for Relation Extraction with Gate Mechanism [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 39-44.
[7]	ZHANG Qinglin, DU Jiachen, XU Ruifeng. Sarcasm Detection Based on Adversarial Learning [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(1): 29-36.
[8]	XU Haoliang, LI Yanqun, HE Yunqi, QIAN Longhua. Research on Chinese Nested Named Entity Relation Extraction [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(1): 8-14.
[9]	LUO Fan, WANG Houfeng. Chinese Text Sentiment Classification by H-RNN-CNN [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2018, 54(3): 459-465.
[10]	Wenhao YING, Xinyan XIAO, Sujian LI, Yajuan LÜ, Zhifang SUI. Improving Query-Focused Summarization with CNN-Based Similarity [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2017, 53(2): 197-203.

Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks

基于可融合残差卷积块的深度神经网络模型层剪枝方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 10

Recommended Articles

Metrics