Softmax regression design for stochastic computing based deep convolutional neural networks

Zihao Yuan, Ji Li, Zhe Li, Caiwen Ding, Ao Ren, Bo Yuan, Qinru Qiu, Jeffrey Draper, Yanzhi Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

Recently, Deep Convolutional Neural Networks (DCNNs) have made tremendous advances, achieving close to or even better accuracy than human-level perception in various tasks. Stochastic Computing (SC), as an alternate to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementations of DCNNs. In this paper, we design and optimize the SC based Softmax Regression function. Experiment results show that compared with a binary SR, the proposed SC-SR under longer bit stream can reach the same level of accuracy with the improvement of 295X, 62X, 2617X in terms of power, area and energy, respectively. Binary SR is suggested for future DCNNs with short bit stream length input whereas SC-SR is recommended for longer bit stream.

Original languageEnglish (US)
Title of host publicationGLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017
PublisherAssociation for Computing Machinery
Pages467-470
Number of pages4
VolumePart F127756
ISBN (Electronic)9781450349727
DOIs
StatePublished - May 10 2017
Event27th Great Lakes Symposium on VLSI, GLSVLSI 2017 - Banff, Canada
Duration: May 10 2017May 12 2017

Other

Other27th Great Lakes Symposium on VLSI, GLSVLSI 2017
CountryCanada
CityBanff
Period5/10/175/12/17

Fingerprint

Neural networks
Hardware
Experiments

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Yuan, Z., Li, J., Li, Z., Ding, C., Ren, A., Yuan, B., ... Wang, Y. (2017). Softmax regression design for stochastic computing based deep convolutional neural networks. In GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017 (Vol. Part F127756, pp. 467-470). Association for Computing Machinery. https://doi.org/10.1145/3060403.3060467

Softmax regression design for stochastic computing based deep convolutional neural networks. / Yuan, Zihao; Li, Ji; Li, Zhe; Ding, Caiwen; Ren, Ao; Yuan, Bo; Qiu, Qinru; Draper, Jeffrey; Wang, Yanzhi.

GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017. Vol. Part F127756 Association for Computing Machinery, 2017. p. 467-470.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yuan, Z, Li, J, Li, Z, Ding, C, Ren, A, Yuan, B, Qiu, Q, Draper, J & Wang, Y 2017, Softmax regression design for stochastic computing based deep convolutional neural networks. in GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017. vol. Part F127756, Association for Computing Machinery, pp. 467-470, 27th Great Lakes Symposium on VLSI, GLSVLSI 2017, Banff, Canada, 5/10/17. https://doi.org/10.1145/3060403.3060467
Yuan Z, Li J, Li Z, Ding C, Ren A, Yuan B et al. Softmax regression design for stochastic computing based deep convolutional neural networks. In GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017. Vol. Part F127756. Association for Computing Machinery. 2017. p. 467-470 https://doi.org/10.1145/3060403.3060467
Yuan, Zihao ; Li, Ji ; Li, Zhe ; Ding, Caiwen ; Ren, Ao ; Yuan, Bo ; Qiu, Qinru ; Draper, Jeffrey ; Wang, Yanzhi. / Softmax regression design for stochastic computing based deep convolutional neural networks. GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017. Vol. Part F127756 Association for Computing Machinery, 2017. pp. 467-470
@inproceedings{0a1d0721647040bda6b352fd65a92eac,
title = "Softmax regression design for stochastic computing based deep convolutional neural networks",
abstract = "Recently, Deep Convolutional Neural Networks (DCNNs) have made tremendous advances, achieving close to or even better accuracy than human-level perception in various tasks. Stochastic Computing (SC), as an alternate to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementations of DCNNs. In this paper, we design and optimize the SC based Softmax Regression function. Experiment results show that compared with a binary SR, the proposed SC-SR under longer bit stream can reach the same level of accuracy with the improvement of 295X, 62X, 2617X in terms of power, area and energy, respectively. Binary SR is suggested for future DCNNs with short bit stream length input whereas SC-SR is recommended for longer bit stream.",
author = "Zihao Yuan and Ji Li and Zhe Li and Caiwen Ding and Ao Ren and Bo Yuan and Qinru Qiu and Jeffrey Draper and Yanzhi Wang",
year = "2017",
month = "5",
day = "10",
doi = "10.1145/3060403.3060467",
language = "English (US)",
volume = "Part F127756",
pages = "467--470",
booktitle = "GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - Softmax regression design for stochastic computing based deep convolutional neural networks

AU - Yuan, Zihao

AU - Li, Ji

AU - Li, Zhe

AU - Ding, Caiwen

AU - Ren, Ao

AU - Yuan, Bo

AU - Qiu, Qinru

AU - Draper, Jeffrey

AU - Wang, Yanzhi

PY - 2017/5/10

Y1 - 2017/5/10

N2 - Recently, Deep Convolutional Neural Networks (DCNNs) have made tremendous advances, achieving close to or even better accuracy than human-level perception in various tasks. Stochastic Computing (SC), as an alternate to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementations of DCNNs. In this paper, we design and optimize the SC based Softmax Regression function. Experiment results show that compared with a binary SR, the proposed SC-SR under longer bit stream can reach the same level of accuracy with the improvement of 295X, 62X, 2617X in terms of power, area and energy, respectively. Binary SR is suggested for future DCNNs with short bit stream length input whereas SC-SR is recommended for longer bit stream.

AB - Recently, Deep Convolutional Neural Networks (DCNNs) have made tremendous advances, achieving close to or even better accuracy than human-level perception in various tasks. Stochastic Computing (SC), as an alternate to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementations of DCNNs. In this paper, we design and optimize the SC based Softmax Regression function. Experiment results show that compared with a binary SR, the proposed SC-SR under longer bit stream can reach the same level of accuracy with the improvement of 295X, 62X, 2617X in terms of power, area and energy, respectively. Binary SR is suggested for future DCNNs with short bit stream length input whereas SC-SR is recommended for longer bit stream.

UR - http://www.scopus.com/inward/record.url?scp=85021227831&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021227831&partnerID=8YFLogxK

U2 - 10.1145/3060403.3060467

DO - 10.1145/3060403.3060467

M3 - Conference contribution

AN - SCOPUS:85021227831

VL - Part F127756

SP - 467

EP - 470

BT - GLSVLSI 2017 - Proceedings of the Great Lakes Symposium on VLSI 2017

PB - Association for Computing Machinery

ER -