An energy-efficient stochastic computational deep belief network

Yidong Liu, Yanzhi Wang, Fabrizio Lombardi, Jie Han

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

Deep neural networks (DNNs) are effective machine learning models to solve a large class of recognition problems, including the classification of nonlinearly separable patterns. The applications of DNNs are, however, limited by the large size and high energy consumption of the networks. Recently, stochastic computation (SC) has been considered to implement DNNs to reduce the hardware cost. However, it requires a large number of random number generators (RNGs) that lower the energy efficiency of the network. To overcome these limitations, we propose the design of an energy-efficient deep belief network (DBN) based on stochastic computation. An approximate SC activation unit (A-SCAU) is designed to implement different types of activation functions in the neurons. The A-SCAU is immune to signal correlations, so the RNGs can be shared among all neurons in the same layer with no accuracy loss. The area and energy of the proposed design are 5.27% and 3.31% (or 26.55% and 29.89%) of a 32-bit floating-point (or an 8-bit fixed-point) implementation. It is shown that the proposed SC-DBN design achieves a higher classification accuracy compared to the fixed-point implementation. The accuracy is only lower by 0.12% than the floating-point design at a similar computation speed, but with a significantly lower energy consumption.

Original languageEnglish (US)
Title of host publicationProceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1175-1178
Number of pages4
Volume2018-January
ISBN (Electronic)9783981926316
DOIs
StatePublished - Apr 19 2018
Event2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018 - Dresden, Germany
Duration: Mar 19 2018Mar 23 2018

Other

Other2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018
CountryGermany
CityDresden
Period3/19/183/23/18

Keywords

  • cognitive computing
  • deep belief network
  • rectifier linear unit
  • stochastic computing

ASJC Scopus subject areas

  • Safety, Risk, Reliability and Quality
  • Hardware and Architecture
  • Software
  • Information Systems and Management

Fingerprint Dive into the research topics of 'An energy-efficient stochastic computational deep belief network'. Together they form a unique fingerprint.

  • Cite this

    Liu, Y., Wang, Y., Lombardi, F., & Han, J. (2018). An energy-efficient stochastic computational deep belief network. In Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018 (Vol. 2018-January, pp. 1175-1178). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/DATE.2018.8342191