SYSTEMATIC WEIGHT PRUNING OF DNNS USING ALTERNATING DIRECTION METHOD OF MULTIPLIERS

Tianyun Zhang; Shaokai Ye; Yipeng Zhang; Yanzhi Wang; Makan Fardad

SYSTEMATIC WEIGHT PRUNING OF DNNS USING ALTERNATING DIRECTION METHOD OF MULTIPLIERS

Tianyun Zhang, Shaokai Ye, Yipeng Zhang, Yanzhi Wang, Makan Fardad

Department of Electrical Engineering & Computer Science

Research output: Contribution to conference › Paper › peer-review

12 Scopus citations

Abstract

We present a systematic weight pruning framework of deep neural networks (DNNs) using the alternating direction method of multipliers (ADMM). We first formulate the weight pruning problem of DNNs as a constrained nonconvex optimization problem, and then adopt the ADMM framework for systematic weight pruning. We show that ADMM is highly suitable for weight pruning due to the computational efficiency it offers. We achieve a much higher compression ratio compared with prior work while maintaining the same test accuracy, together with a faster convergence rate.

Original language	English (US)
State	Published - 2018
Event	6th International Conference on Learning Representations, ICLR 2018 - Vancouver, Canada Duration: Apr 30 2018 → May 3 2018

Conference

Conference	6th International Conference on Learning Representations, ICLR 2018
Country/Territory	Canada
City	Vancouver
Period	4/30/18 → 5/3/18

ASJC Scopus subject areas

Education
Computer Science Applications
Linguistics and Language
Language and Linguistics

Cite this

@conference{6094a4ef247740eaa386285292fb74e1,

title = "SYSTEMATIC WEIGHT PRUNING OF DNNS USING ALTERNATING DIRECTION METHOD OF MULTIPLIERS",

abstract = "We present a systematic weight pruning framework of deep neural networks (DNNs) using the alternating direction method of multipliers (ADMM). We first formulate the weight pruning problem of DNNs as a constrained nonconvex optimization problem, and then adopt the ADMM framework for systematic weight pruning. We show that ADMM is highly suitable for weight pruning due to the computational efficiency it offers. We achieve a much higher compression ratio compared with prior work while maintaining the same test accuracy, together with a faster convergence rate.",

author = "Tianyun Zhang and Shaokai Ye and Yipeng Zhang and Yanzhi Wang and Makan Fardad",

note = "Publisher Copyright: {\textcopyright} 2017 International Conference on Learning Representations, ICLR. All rights reserved.; 6th International Conference on Learning Representations, ICLR 2018 ; Conference date: 30-04-2018 Through 03-05-2018",

year = "2018",

language = "English (US)",

}

TY - CONF

T1 - SYSTEMATIC WEIGHT PRUNING OF DNNS USING ALTERNATING DIRECTION METHOD OF MULTIPLIERS

AU - Zhang, Tianyun

AU - Ye, Shaokai

AU - Zhang, Yipeng

AU - Wang, Yanzhi

AU - Fardad, Makan

PY - 2018

Y1 - 2018

N2 - We present a systematic weight pruning framework of deep neural networks (DNNs) using the alternating direction method of multipliers (ADMM). We first formulate the weight pruning problem of DNNs as a constrained nonconvex optimization problem, and then adopt the ADMM framework for systematic weight pruning. We show that ADMM is highly suitable for weight pruning due to the computational efficiency it offers. We achieve a much higher compression ratio compared with prior work while maintaining the same test accuracy, together with a faster convergence rate.

AB - We present a systematic weight pruning framework of deep neural networks (DNNs) using the alternating direction method of multipliers (ADMM). We first formulate the weight pruning problem of DNNs as a constrained nonconvex optimization problem, and then adopt the ADMM framework for systematic weight pruning. We show that ADMM is highly suitable for weight pruning due to the computational efficiency it offers. We achieve a much higher compression ratio compared with prior work while maintaining the same test accuracy, together with a faster convergence rate.

UR - http://www.scopus.com/inward/record.url?scp=85169792944&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85169792944&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85169792944

T2 - 6th International Conference on Learning Representations, ICLR 2018

Y2 - 30 April 2018 through 3 May 2018

ER -

SYSTEMATIC WEIGHT PRUNING OF DNNS USING ALTERNATING DIRECTION METHOD OF MULTIPLIERS

Abstract

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this