Towards acceleration of deep convolutional neural networks using stochastic computing

Ji Li, Ao Ren, Zhe Li, Caiwen Ding, Bo Yuan, Qinru Qiu, Yanzhi Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

21 Citations (Scopus)

Abstract

In recent years, Deep Convolutional Neural Network (DCNN) has become the dominant approach for almost all recognition and detection tasks and outperformed humans on certain tasks. Nevertheless, the high power consumptions and complex topologies have hindered the widespread deployment of DCNNs, particularly in wearable devices and embedded systems with limited area and power budget. This paper presents a fully parallel and scalable hardware-based DCNN design using Stochastic Computing (SC), which leverages the energy-accuracy trade-off through optimizing SC components in different layers. We first conduct a detailed investigation of the Approximate Parallel Counter (APC) based neuron and multiplexer-based neuron using SC, and analyze the impacts of various design parameters, such as bit stream length and input number, on the energy/power/area/accuracy of the neuron cell. Then, from an architecture perspective, the influence of inaccuracy of neurons in different layers on the overall DCNN accuracy (i.e., software accuracy of the entire DCNN) is studied. Accordingly, a structure optimization method is proposed for a general DCNN architecture, in which neurons in different layers are implemented with optimized SC components, so as to reduce the area, power, and energy of the DCNN while maintaining the overall network performance in terms of accuracy. Experimental results show that the proposed approach can find a satisfactory DCNN configuration, which achieves 55X, 151X, and 2X improvement in terms of area, power and energy, respectively, while the error is increased by 2.86%, compared with the conventional binary ASIC implementation.

Original languageEnglish (US)
Title of host publication2017 22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages115-120
Number of pages6
ISBN (Electronic)9781509015580
DOIs
StatePublished - Feb 16 2017
Event22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017 - Chiba, Japan
Duration: Jan 16 2017Jan 19 2017

Other

Other22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017
CountryJapan
CityChiba
Period1/16/171/19/17

Fingerprint

Neural networks
Neurons
Application specific integrated circuits
Network performance
Network architecture
Embedded systems
Electric power utilization
Topology
Hardware

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Computer Science Applications
  • Computer Graphics and Computer-Aided Design

Cite this

Li, J., Ren, A., Li, Z., Ding, C., Yuan, B., Qiu, Q., & Wang, Y. (2017). Towards acceleration of deep convolutional neural networks using stochastic computing. In 2017 22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017 (pp. 115-120). [7858306] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ASPDAC.2017.7858306

Towards acceleration of deep convolutional neural networks using stochastic computing. / Li, Ji; Ren, Ao; Li, Zhe; Ding, Caiwen; Yuan, Bo; Qiu, Qinru; Wang, Yanzhi.

2017 22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 115-120 7858306.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Li, J, Ren, A, Li, Z, Ding, C, Yuan, B, Qiu, Q & Wang, Y 2017, Towards acceleration of deep convolutional neural networks using stochastic computing. in 2017 22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017., 7858306, Institute of Electrical and Electronics Engineers Inc., pp. 115-120, 22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017, Chiba, Japan, 1/16/17. https://doi.org/10.1109/ASPDAC.2017.7858306
Li J, Ren A, Li Z, Ding C, Yuan B, Qiu Q et al. Towards acceleration of deep convolutional neural networks using stochastic computing. In 2017 22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 115-120. 7858306 https://doi.org/10.1109/ASPDAC.2017.7858306
Li, Ji ; Ren, Ao ; Li, Zhe ; Ding, Caiwen ; Yuan, Bo ; Qiu, Qinru ; Wang, Yanzhi. / Towards acceleration of deep convolutional neural networks using stochastic computing. 2017 22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 115-120
@inproceedings{fe39b9b164e140099609c46746ed8c0f,
title = "Towards acceleration of deep convolutional neural networks using stochastic computing",
abstract = "In recent years, Deep Convolutional Neural Network (DCNN) has become the dominant approach for almost all recognition and detection tasks and outperformed humans on certain tasks. Nevertheless, the high power consumptions and complex topologies have hindered the widespread deployment of DCNNs, particularly in wearable devices and embedded systems with limited area and power budget. This paper presents a fully parallel and scalable hardware-based DCNN design using Stochastic Computing (SC), which leverages the energy-accuracy trade-off through optimizing SC components in different layers. We first conduct a detailed investigation of the Approximate Parallel Counter (APC) based neuron and multiplexer-based neuron using SC, and analyze the impacts of various design parameters, such as bit stream length and input number, on the energy/power/area/accuracy of the neuron cell. Then, from an architecture perspective, the influence of inaccuracy of neurons in different layers on the overall DCNN accuracy (i.e., software accuracy of the entire DCNN) is studied. Accordingly, a structure optimization method is proposed for a general DCNN architecture, in which neurons in different layers are implemented with optimized SC components, so as to reduce the area, power, and energy of the DCNN while maintaining the overall network performance in terms of accuracy. Experimental results show that the proposed approach can find a satisfactory DCNN configuration, which achieves 55X, 151X, and 2X improvement in terms of area, power and energy, respectively, while the error is increased by 2.86{\%}, compared with the conventional binary ASIC implementation.",
author = "Ji Li and Ao Ren and Zhe Li and Caiwen Ding and Bo Yuan and Qinru Qiu and Yanzhi Wang",
year = "2017",
month = "2",
day = "16",
doi = "10.1109/ASPDAC.2017.7858306",
language = "English (US)",
pages = "115--120",
booktitle = "2017 22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Towards acceleration of deep convolutional neural networks using stochastic computing

AU - Li, Ji

AU - Ren, Ao

AU - Li, Zhe

AU - Ding, Caiwen

AU - Yuan, Bo

AU - Qiu, Qinru

AU - Wang, Yanzhi

PY - 2017/2/16

Y1 - 2017/2/16

N2 - In recent years, Deep Convolutional Neural Network (DCNN) has become the dominant approach for almost all recognition and detection tasks and outperformed humans on certain tasks. Nevertheless, the high power consumptions and complex topologies have hindered the widespread deployment of DCNNs, particularly in wearable devices and embedded systems with limited area and power budget. This paper presents a fully parallel and scalable hardware-based DCNN design using Stochastic Computing (SC), which leverages the energy-accuracy trade-off through optimizing SC components in different layers. We first conduct a detailed investigation of the Approximate Parallel Counter (APC) based neuron and multiplexer-based neuron using SC, and analyze the impacts of various design parameters, such as bit stream length and input number, on the energy/power/area/accuracy of the neuron cell. Then, from an architecture perspective, the influence of inaccuracy of neurons in different layers on the overall DCNN accuracy (i.e., software accuracy of the entire DCNN) is studied. Accordingly, a structure optimization method is proposed for a general DCNN architecture, in which neurons in different layers are implemented with optimized SC components, so as to reduce the area, power, and energy of the DCNN while maintaining the overall network performance in terms of accuracy. Experimental results show that the proposed approach can find a satisfactory DCNN configuration, which achieves 55X, 151X, and 2X improvement in terms of area, power and energy, respectively, while the error is increased by 2.86%, compared with the conventional binary ASIC implementation.

AB - In recent years, Deep Convolutional Neural Network (DCNN) has become the dominant approach for almost all recognition and detection tasks and outperformed humans on certain tasks. Nevertheless, the high power consumptions and complex topologies have hindered the widespread deployment of DCNNs, particularly in wearable devices and embedded systems with limited area and power budget. This paper presents a fully parallel and scalable hardware-based DCNN design using Stochastic Computing (SC), which leverages the energy-accuracy trade-off through optimizing SC components in different layers. We first conduct a detailed investigation of the Approximate Parallel Counter (APC) based neuron and multiplexer-based neuron using SC, and analyze the impacts of various design parameters, such as bit stream length and input number, on the energy/power/area/accuracy of the neuron cell. Then, from an architecture perspective, the influence of inaccuracy of neurons in different layers on the overall DCNN accuracy (i.e., software accuracy of the entire DCNN) is studied. Accordingly, a structure optimization method is proposed for a general DCNN architecture, in which neurons in different layers are implemented with optimized SC components, so as to reduce the area, power, and energy of the DCNN while maintaining the overall network performance in terms of accuracy. Experimental results show that the proposed approach can find a satisfactory DCNN configuration, which achieves 55X, 151X, and 2X improvement in terms of area, power and energy, respectively, while the error is increased by 2.86%, compared with the conventional binary ASIC implementation.

UR - http://www.scopus.com/inward/record.url?scp=85015332539&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85015332539&partnerID=8YFLogxK

U2 - 10.1109/ASPDAC.2017.7858306

DO - 10.1109/ASPDAC.2017.7858306

M3 - Conference contribution

AN - SCOPUS:85015332539

SP - 115

EP - 120

BT - 2017 22nd Asia and South Pacific Design Automation Conference, ASP-DAC 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -