Softmax regression design for stochastic computing based deep convolutional neural networks

Zihao Yuan; Ji Li; Zhe Li; Caiwen Ding; Ao Ren; Bo Yuan; Qinru Qiu; Jeffrey Draper; Yanzhi Wang

doi:10.1145/3060403.3060467

Softmax regression design for stochastic computing based deep convolutional neural networks

Zihao Yuan, Ji Li, Zhe Li, Caiwen Ding, Ao Ren, Bo Yuan, Qinru Qiu, Jeffrey Draper, Yanzhi Wang

Department of Electrical Engineering & Computer Science

Research output: Chapter in Book/Entry/Poem › Conference contribution

16 Scopus citations

Abstract

Recently, Deep Convolutional Neural Networks (DCNNs) have made tremendous advances, achieving close to or even better accuracy than human-level perception in various tasks. Stochastic Computing (SC), as an alternate to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementations of DCNNs. In this paper, we design and optimize the SC based Softmax Regression function. Experiment results show that compared with a binary SR, the proposed SC-SR under longer bit stream can reach the same level of accuracy with the improvement of 295X, 62X, 2617X in terms of power, area and energy, respectively. Binary SR is suggested for future DCNNs with short bit stream length input whereas SC-SR is recommended for longer bit stream.

Original language	English (US)
Title of host publication	GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017
Publisher	Association for Computing Machinery
Pages	467-470
Number of pages	4
ISBN (Electronic)	9781450349727
DOIs	https://doi.org/10.1145/3060403.3060467
State	Published - May 10 2017
Event	27th Great Lakes Symposium on VLSI, GLSVLSI 2017 - Banff, Canada Duration: May 10 2017 → May 12 2017

Publication series

Name	Proceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI
Volume	Part F127756

Other

Other	27th Great Lakes Symposium on VLSI, GLSVLSI 2017
Country/Territory	Canada
City	Banff
Period	5/10/17 → 5/12/17

ASJC Scopus subject areas

General Engineering

Access to Document

10.1145/3060403.3060467

Cite this

Yuan, Z., Li, J., Li, Z., Ding, C., Ren, A., Yuan, B., Qiu, Q., Draper, J., & Wang, Y. (2017). Softmax regression design for stochastic computing based deep convolutional neural networks. In GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017 (pp. 467-470). (Proceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI; Vol. Part F127756). Association for Computing Machinery. https://doi.org/10.1145/3060403.3060467

Softmax regression design for stochastic computing based deep convolutional neural networks. / Yuan, Zihao; Li, Ji; Li, Zhe et al.
GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017. Association for Computing Machinery, 2017. p. 467-470 (Proceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI; Vol. Part F127756).

Research output: Chapter in Book/Entry/Poem › Conference contribution

Yuan, Z, Li, J, Li, Z, Ding, C, Ren, A, Yuan, B, Qiu, Q, Draper, J & Wang, Y 2017, Softmax regression design for stochastic computing based deep convolutional neural networks. in GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017. Proceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI, vol. Part F127756, Association for Computing Machinery, pp. 467-470, 27th Great Lakes Symposium on VLSI, GLSVLSI 2017, Banff, Canada, 5/10/17. https://doi.org/10.1145/3060403.3060467

@inproceedings{0a1d0721647040bda6b352fd65a92eac,

title = "Softmax regression design for stochastic computing based deep convolutional neural networks",

abstract = "Recently, Deep Convolutional Neural Networks (DCNNs) have made tremendous advances, achieving close to or even better accuracy than human-level perception in various tasks. Stochastic Computing (SC), as an alternate to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementations of DCNNs. In this paper, we design and optimize the SC based Softmax Regression function. Experiment results show that compared with a binary SR, the proposed SC-SR under longer bit stream can reach the same level of accuracy with the improvement of 295X, 62X, 2617X in terms of power, area and energy, respectively. Binary SR is suggested for future DCNNs with short bit stream length input whereas SC-SR is recommended for longer bit stream.",

author = "Zihao Yuan and Ji Li and Zhe Li and Caiwen Ding and Ao Ren and Bo Yuan and Qinru Qiu and Jeffrey Draper and Yanzhi Wang",

note = "Publisher Copyright: {\textcopyright} 2017 ACM.; 27th Great Lakes Symposium on VLSI, GLSVLSI 2017 ; Conference date: 10-05-2017 Through 12-05-2017",

year = "2017",

month = may,

day = "10",

doi = "10.1145/3060403.3060467",

language = "English (US)",

series = "Proceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI",

publisher = "Association for Computing Machinery",

pages = "467--470",

booktitle = "GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017",

}

TY - GEN

T1 - Softmax regression design for stochastic computing based deep convolutional neural networks

AU - Yuan, Zihao

AU - Li, Ji

AU - Li, Zhe

AU - Ding, Caiwen

AU - Ren, Ao

AU - Yuan, Bo

AU - Qiu, Qinru

AU - Draper, Jeffrey

AU - Wang, Yanzhi

PY - 2017/5/10

Y1 - 2017/5/10

N2 - Recently, Deep Convolutional Neural Networks (DCNNs) have made tremendous advances, achieving close to or even better accuracy than human-level perception in various tasks. Stochastic Computing (SC), as an alternate to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementations of DCNNs. In this paper, we design and optimize the SC based Softmax Regression function. Experiment results show that compared with a binary SR, the proposed SC-SR under longer bit stream can reach the same level of accuracy with the improvement of 295X, 62X, 2617X in terms of power, area and energy, respectively. Binary SR is suggested for future DCNNs with short bit stream length input whereas SC-SR is recommended for longer bit stream.

AB - Recently, Deep Convolutional Neural Networks (DCNNs) have made tremendous advances, achieving close to or even better accuracy than human-level perception in various tasks. Stochastic Computing (SC), as an alternate to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementations of DCNNs. In this paper, we design and optimize the SC based Softmax Regression function. Experiment results show that compared with a binary SR, the proposed SC-SR under longer bit stream can reach the same level of accuracy with the improvement of 295X, 62X, 2617X in terms of power, area and energy, respectively. Binary SR is suggested for future DCNNs with short bit stream length input whereas SC-SR is recommended for longer bit stream.

UR - http://www.scopus.com/inward/record.url?scp=85021227831&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021227831&partnerID=8YFLogxK

U2 - 10.1145/3060403.3060467

DO - 10.1145/3060403.3060467

M3 - Conference contribution

AN - SCOPUS:85021227831

T3 - Proceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI

SP - 467

EP - 470

BT - GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017

PB - Association for Computing Machinery

T2 - 27th Great Lakes Symposium on VLSI, GLSVLSI 2017

Y2 - 10 May 2017 through 12 May 2017

ER -

Softmax regression design for stochastic computing based deep convolutional neural networks

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this