Model compression method combining multi-factor channel pruning and knowledge distillation

Tao Xie; Xiangqiang Zhao; Yingjie Ma; Le Chang; Chongchong Yu; Wenbin Feng

doi:10.1117/12.2660087

3 February 2023 Model compression method combining multi-factor channel pruning and knowledge distillation

Tao Xie, Xiangqiang Zhao, Yingjie Ma, Le Chang, Chongchong Yu, Wenbin Feng

Proceedings Volume 12511, Third International Conference on Computer Vision and Data Mining (ICCVDM 2022); 125112R (2023) https://doi.org/10.1117/12.2660087
Event: Third International Conference on Computer Vision and Data Mining (ICCVDM 2022), 2022, Hulun Buir, China

Abstract

Deploying deep learning models in embedded terminals is essential for applications with real-time reasoning requirements. In order to make the model run efficiently in the embedded end with limited resources, we propose a model compression method combining multi-factor channel pruning and knowledge distillation. In the process of network sparsity, this method uses the double factors of the BN layer to improve the pruning standard and guides the local pruning of the model according to the new standard to ensure the compression rate. In order to further improve the accuracy, we use the knowledge transfer method in the idea of knowledge distillation to fine-tune the model and use the more continuous parameter distribution of the student model to ensure accuracy. We use several deep learning models to test. The experimental results show that the proposed model compression method has the advantages of fewer parameters and higher accuracy, which reduces the resources occupied by the model application to the embedded end. More importantly, this method not only realizes the efficient operation of various models in the embedded end, but also ensures the high accuracy of the model.

Citation Download Citation

Tao Xie, Xiangqiang Zhao, Yingjie Ma, Le Chang, Chongchong Yu, and Wenbin Feng "Model compression method combining multi-factor channel pruning and knowledge distillation", Proc. SPIE 12511, Third International Conference on Computer Vision and Data Mining (ICCVDM 2022), 125112R (3 February 2023); https://doi.org/10.1117/12.2660087

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
12 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Data modeling

Performance modeling

Convolution

Process modeling

Feature extraction

Target detection

Visual process modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years