DSCNN: Hardware-oriented optimization for Stochastic Computing based Deep Convolutional Neural Networks

Zhe Li, Ao Ren, Ji Li, Qinru Qiu, Yanzhi Wang, Bo Yuan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

28 Scopus citations

Abstract

Deep Convolutional Neural Networks (DCNN), a branch of Deep Neural Networks which use the deep graph with multiple processing layers, enables the convolutional model to finely abstract the high-level features behind an image. Large-scale applications using DCNN mainly operate in high-performance server clusters, GPUs or FPGA clusters; it is restricted to extend the applications onto mobile/wearable devices and Internet-of-Things (IoT) entities due to high power/energy consumption. Stochastic Computing is a promising method to overcome this shortcoming used in specific hardware-based systems. Many complex arithmetic operations can be implemented with very simple hardware logic in the SC framework, which alleviates the extensive computation complexity. The exploration of network-wise optimization and the revision of network structure with respect to stochastic computing based hardware design have not been discussed in previous work. In this paper, we investigate Deep Stochastic Convolutional Neural Network (DSCNN) for DCNN using stochastic computing. The essential calculation components using SC are designed and evaluated. We propose a joint optimization method to collaborate components guaranteeing a high calculation accuracy in each stage of the network. The structure of original DSCNN is revised to accommodate SC hardware design's simplicity. Experimental Results show that as opposed to software inspired feature extraction block in DSCNN, an optimized hardware oriented feature extraction block achieves as higher as 59.27% calculation precision. And the optimized DSCNN can achieve only 3.48% network test error rate compared to 27.83% for baseline DSCNN using software inspired feature extraction block.

Original languageEnglish (US)
Title of host publicationProceedings of the 34th IEEE International Conference on Computer Design, ICCD 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages678-681
Number of pages4
ISBN (Electronic)9781509051427
DOIs
StatePublished - Nov 22 2016
Event34th IEEE International Conference on Computer Design, ICCD 2016 - Scottsdale, United States
Duration: Oct 2 2016Oct 5 2016

Other

Other34th IEEE International Conference on Computer Design, ICCD 2016
CountryUnited States
CityScottsdale
Period10/2/1610/5/16

    Fingerprint

Keywords

  • Deep Convolutional Neural Networks
  • Deep Learning
  • Hardware-oriented Co-optimization
  • Stochastic Computing

ASJC Scopus subject areas

  • Hardware and Architecture

Cite this

Li, Z., Ren, A., Li, J., Qiu, Q., Wang, Y., & Yuan, B. (2016). DSCNN: Hardware-oriented optimization for Stochastic Computing based Deep Convolutional Neural Networks. In Proceedings of the 34th IEEE International Conference on Computer Design, ICCD 2016 (pp. 678-681). [7753357] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCD.2016.7753357