Designing reconfigurable large-scale deep learning systems using stochastic computing

Ao Ren, Zhe Li, Yanzhi Wang, Qinru Qiu, Bo Yuan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Scopus citations

Abstract

Deep Learning, as an important branch of machine learning and neural network, is playing an increasingly important role in a number of fields like computer vision, natural language processing, etc. However, large-scale deep learning systems mainly operate in high-performance server clusters, thus restricting the application extensions to personal or mobile devices. The solution proposed in this paper is taking advantage of the fantastic features of stochastic computing methods. Stochastic computing is a type of data representation and processing technique, which uses a binary bit stream to represent a probability number (by counting the number of ones in this bit stream). In the stochastic computing area, some key arithmetic operations such as additions or multiplications can be implemented with very simple components like AND gates or multiplexers, respectively. Thus it provides an immense design space for integrating a large amount of neurons and enabling fully parallel and scalable hardware implementations of large-scale deep learning systems. In this paper, we present a reconfigurable large-scale deep learning system based on stochastic computing technologies, including the design of the neuron, the convolution function, the back-propagation function and some other basic operations. And the network-on-chip technique is also proposed in this paper to achieve the goal of implementing a large-scale hardware system. Our experiments validate the functionality of reconfigurable deep learning systems using stochastic computing, and demonstrate that when the bit streams are set to be 8192 bits, classification of MNIST digits by stochastic computing can perform as low error rate as that by normal arithmetic operations.

Original languageEnglish (US)
Title of host publication2016 IEEE International Conference on Rebooting Computing, ICRC 2016 - Conference Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509013708
DOIs
StatePublished - Nov 8 2016
Event2016 IEEE International Conference on Rebooting Computing, ICRC 2016 - San Diego, United States
Duration: Oct 17 2016Oct 19 2016

Other

Other2016 IEEE International Conference on Rebooting Computing, ICRC 2016
CountryUnited States
CitySan Diego
Period10/17/1610/19/16

    Fingerprint

Keywords

  • deep learning
  • large-scale
  • neuron
  • reconfigurable
  • Stochastic computing

ASJC Scopus subject areas

  • Hardware and Architecture

Cite this

Ren, A., Li, Z., Wang, Y., Qiu, Q., & Yuan, B. (2016). Designing reconfigurable large-scale deep learning systems using stochastic computing. In 2016 IEEE International Conference on Rebooting Computing, ICRC 2016 - Conference Proceedings [7738685] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICRC.2016.7738685