AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance

doi:10.13209/j.0479-8023.2022.115

Acta Scientiarum Naturalium Universitatis Pekinensis ›› 2023, Vol. 59 ›› Issue (5): 764-772.DOI: 10.13209/j.0479-8023.2022.115

Previous Articles Next Articles

AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance

LIU Xiangcheng¹, CAO Jian^1,†, YAO Hongyi¹, XU Pengtao², ZHANG Yuan¹, WANG Yuan³

1. School of Software and Microelectronics, Peking University, Beijing 102600 2. Xi’an Nuclear Instrument Co., Ltd. Xi’an 710061 3. School of Integrated Circuits, Peking University, Beijing 100871

Received:2022-09-14 Revised:2022-11-11 Online:2023-09-20 Published:2023-09-18
Contact: CAO Jian, E-mail: caojian(at)ss.pku.edu.cn

基于自适应剪枝率与高效权重继承的神经网络通道剪枝方法

刘相呈¹, 曹健^1,†, 姚宏毅¹, 徐鹏涛², 张袁¹, 王源³

1. 北京大学软件与微电子学院, 北京 102600 2. 西安中核核仪器股份有限公司, 西安 710061 3. 北京大学集成电路学院, 北京 100871

通讯作者: 曹健, E-mail: caojian(at)ss.pku.edu.cn
基金资助:
国家重点研发计划(2018YFE0203801)资助

Abstract

Abstract:

Previous channel pruning methods require complex search and fine-tuning processes and are prone to fall into local optimal solutions. To solve this problem, the authors propose a novel channel pruning framework AdaPruner, which can generate corresponding sub-networks adaptively for various budget complexities and efficiently select the initialization weights suitable for the current structure by sparse training once. Experimental results show that the proposed method achieves better performance than previous pruning methods on both commonly used residual networks and lightweight networks on multiple datasets for image classification task.

Key words: convolutional neural network, channel pruning, sparse training, neural network architecture search; image classification

摘要：

目前的通道级剪枝方法往往需要复杂的搜索和微调过程, 并且容易陷入局部最优解, 针对此问题, 提出一种新颖的通道剪枝框架(AdaPruner), 只需通过一次稀疏训练, 就可以针对各种预算复杂度, 自适应地生成相应的子网络, 并高效地选择适合当前结构的初始化权重。在图像分类任务的多个数据集上实验结果表明, 该方法在常用的残差网络和轻量级网络上的性能都优于以往剪枝方法。

关键词: 卷积神经网络, 通道剪枝, 稀疏化训练, 神经网络结构搜索, 图像分类

LIU Xiangcheng, CAO Jian, YAO Hongyi, XU Pengtao, ZHANG Yuan, WANG Yuan. AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(5): 764-772.

刘相呈, 曹健, 姚宏毅, 徐鹏涛, 张袁, 王源. 基于自适应剪枝率与高效权重继承的神经网络通道剪枝方法[J]. 北京大学学报自然科学版, 2023, 59(5): 764-772.

[1]	ZHOU Xiangyu, MAO Shanjun, LI Mei. Bearing Fault Diagnosis Method Based on Down-Sampling in Frequency Domain and CNN [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2023, 59(2): 251-260.
[2]	XIE Hao, CAO Jian, LI Pu, ZHAO Xiongbo, ZHANG Xing. A Hardware Accelerator for SSD Object Detection Algorithm Based on FPGA [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(6): 1015-1022.
[3]	XU Pengtao, CAO Jian, CHEN Weiqian, LIU Shengrong, WANG Yuan, ZHANG Xing. Post Training Quantization Preprocessing Method of Convolutional Neural Network via Outlier Removal [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(5): 808-812.
[4]	XU Pengtao, CAO Jian, SUN Wenyu, LI Pu, WANG Yuan, ZHANG Xing. Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2022, 58(5): 801-807.
[5]	SUN Wenyu, CAO Jian, LI Pu, LIU Rui. Pruning and Fine-tuning Optimization Method of Convolutional Neural Network Based on Global Information [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(4): 790-794.
[6]	LI Xingya, CHEN Yufeng, XU Jin’an, ZHANG Yujie. Distant Supervision for Relation Extraction with Gate Mechanism [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1): 39-44.
[7]	ZHANG Qinglin, DU Jiachen, XU Ruifeng. Sarcasm Detection Based on Adversarial Learning [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(1): 29-36.
[8]	XU Haoliang, LI Yanqun, HE Yunqi, QIAN Longhua. Research on Chinese Nested Named Entity Relation Extraction [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019, 55(1): 8-14.
[9]	LUO Fan, WANG Houfeng. Chinese Text Sentiment Classification by H-RNN-CNN [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2018, 54(3): 459-465.
[10]	Wenhao YING, Xinyan XIAO, Sujian LI, Yajuan LÜ, Zhifang SUI. Improving Query-Focused Summarization with CNN-Based Similarity [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2017, 53(2): 197-203.

AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance

基于自适应剪枝率与高效权重继承的神经网络通道剪枝方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 10

Recommended Articles

Metrics