Structural design optimization for deep convolutional neural networks using stochastic computing

Zhe Li, Ao Ren, Ji Li, Qinru Qiu, Bo Yuan, Jeffrey Draper, Yanzhi Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Scopus citations

Abstract

Deep Convolutional Neural Networks (DCNNs) have been demonstrated as effective models for understanding image content. The computation behind DCNNs highly relies on the capability of hardware resources due to the deep structure. DCNNs have been implemented on different large-scale computing platforms. However, there is a trend that DCNNs have been embedded into light-weight local systems, which requires low power/energy consumptions and small hardware footprints. Stochastic Computing (SC) radically simplifies the hardware implementation of arithmetic units and has the potential to satisfy the small low-power needs of DCNNs. Local connectivities and down-sampling operations have made DCNNs more complex to be implemented using SC. In this paper, eight feature extraction designs for DCNNs using SC in two groups are explored and optimized in detail from the perspective of calculation precision, where we permute two SC implementations for inner-product calculation, two down-sampling schemes, and two structures of DCNN neurons. We evaluate the network in aspects of network accuracy and hardware performance for each DCNN using one feature extraction design out of eight. Through exploration and optimization, the accuracies of SC-based DCNNs are guaranteed compared with software implementations on CPU/GPU/binary-based ASIC synthesis, while area, power, and energy are significantly reduced by up to 776x, 190x, and 32835x.

Original languageEnglish (US)
Title of host publicationProceedings of the 2017 Design, Automation and Test in Europe, DATE 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages250-253
Number of pages4
ISBN (Electronic)9783981537093
DOIs
StatePublished - May 11 2017
Event20th Design, Automation and Test in Europe, DATE 2017 - Swisstech, Lausanne, Switzerland
Duration: Mar 27 2017Mar 31 2017

Other

Other20th Design, Automation and Test in Europe, DATE 2017
CountrySwitzerland
CitySwisstech, Lausanne
Period3/27/173/31/17

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'Structural design optimization for deep convolutional neural networks using stochastic computing'. Together they form a unique fingerprint.

  • Cite this

    Li, Z., Ren, A., Li, J., Qiu, Q., Yuan, B., Draper, J., & Wang, Y. (2017). Structural design optimization for deep convolutional neural networks using stochastic computing. In Proceedings of the 2017 Design, Automation and Test in Europe, DATE 2017 (pp. 250-253). [7926991] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/DATE.2017.7926991