A Unified DNN Weight Pruning Framework Using Reweighted Optimization Methods

Tianyun Zhang, Xiaolong Ma, Zheng Zhan, Shanglin Zhou, Caiwen Ding, Makan Fardad, Yanzhi Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

To address the large model size and intensive computation requirement of deep neural networks (DNNs), weight pruning techniques have been proposed and generally fall into two categories, i.e., static regularization-based pruning and dynamic regularization-based pruning. However, the former method currently suffers either complex workloads or accuracy degradation, while the latter one takes a long time to tune the parameters to achieve the desired pruning rate without accuracy loss. In this paper, we propose a unified DNN weight pruning framework with dynamically updated regularization terms bounded by the designated constraint. Our proposed method increases the compression rate, reduces the training time and reduces the number of hyper-parameters compared with state-of-the-art ADMM-based hard constraint method.

Original languageEnglish (US)
Title of host publication2021 58th ACM/IEEE Design Automation Conference, DAC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages493-498
Number of pages6
ISBN (Electronic)9781665432740
DOIs
StatePublished - Dec 5 2021
Event58th ACM/IEEE Design Automation Conference, DAC 2021 - San Francisco, United States
Duration: Dec 5 2021Dec 9 2021

Publication series

NameProceedings - Design Automation Conference
Volume2021-December
ISSN (Print)0738-100X

Conference

Conference58th ACM/IEEE Design Automation Conference, DAC 2021
Country/TerritoryUnited States
CitySan Francisco
Period12/5/2112/9/21

ASJC Scopus subject areas

  • Computer Science Applications
  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Modeling and Simulation

Fingerprint

Dive into the research topics of 'A Unified DNN Weight Pruning Framework Using Reweighted Optimization Methods'. Together they form a unique fingerprint.

Cite this