Hardware acceleration of Bayesian neural networks using RAM based linear feedback Gaussian random number generators

Ruizhe Cai, Ao Ren, Luhao Wangy, Massoud Pedramy, Yanzhi Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Bayesian neural networks (BNNs) have been proposed to address the problem of model uncertainty in training. By introducing weights associated with conditioned probability distributions, BNN is capable to resolve overfitting issues commonly seen in conventional neural networks. Frequent usage of Gaussian random variables requires a properly optimized Gaussian Random Number Generator (GRNG). The high hardware cost of conventional GRNG makes the hardware realization of BNN challenging. In this paper, a new hardware acceleration architecture for variational inference in BNNs is proposed to facilitate the applicability of BNN in larger-scale applications. In addition, the proposed implementation introduced the RAM based Linear Feedback based GRNG (RLF-GRNG) for effective weight sampling in BNNs. The RAM based Linear Feedback method can effectively utilize RAM resources for parallel Gaussian random number generation while requiring limited and sharable control logic. Implementation on an Altera Cyclone V FPGA suggests that the RLF-GRNG utilizes much less RAM resources compared to other GRNG methods. Experiments results show that the proposed hardware implementation of a BNN can still attain similar accuracy compared to software implementation.

Original languageEnglish (US)
Title of host publicationProceedings - 35th IEEE International Conference on Computer Design, ICCD 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages289-296
Number of pages8
ISBN (Electronic)9781538622544
DOIs
StatePublished - Nov 22 2017
Event35th IEEE International Conference on Computer Design, ICCD 2017 - Boston, United States
Duration: Nov 5 2017Nov 8 2017

Publication series

NameProceedings - 35th IEEE International Conference on Computer Design, ICCD 2017

Other

Other35th IEEE International Conference on Computer Design, ICCD 2017
CountryUnited States
CityBoston
Period11/5/1711/8/17

ASJC Scopus subject areas

  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Hardware acceleration of Bayesian neural networks using RAM based linear feedback Gaussian random number generators'. Together they form a unique fingerprint.

  • Cite this

    Cai, R., Ren, A., Wangy, L., Pedramy, M., & Wang, Y. (2017). Hardware acceleration of Bayesian neural networks using RAM based linear feedback Gaussian random number generators. In Proceedings - 35th IEEE International Conference on Computer Design, ICCD 2017 (pp. 289-296). [8119224] (Proceedings - 35th IEEE International Conference on Computer Design, ICCD 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCD.2017.51