Accelerating Block-Circulant Matrix-Based Neural Network Layer on a General Purpose Computing Platform: A Design Guideline

Krittaphat Pugdeethosapol, Zhao Jin, Daniel Rider, Qinru Qiu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep neural networks (DNNs) have become a powerful tool and enabled the state-of-the art accuracy on many challenging tasks. However, large-scale DNNs highly consume both computational time and storage space. To optimize and improve the performance of the network while maintaining the accuracy, the block-circulant matrix-based (BCM) algorithm has been introduced. BCM utilizes the Fast Fourier Transform (FFT) with block-circulant matrices to compute the output of each layer of the network. Unlike conventional pruning techniques, the network structure is maintained while using the BCM. Compared to conventional matrix implementation, the BCM reduces the computational complexity of a neural network layer from O(n^2) to O(n^2/k), and it has been proven to be highly effective when implemented using customized hardware, such as FPGAs. However, its performance suffers from overhead of FFT and matrix reshaping on general purpose computing platforms. In certain cases, using the BCM does not improve the total computation time of the networks at all. In this paper, we propose a parallel implementation of the BCM layer and guidelines that generally lead to better implementation practice is provided. The guidelines run across popular implementation language and packages including Python, numpy, intel-numpy, tensorflow, and nGraph.

Original languageEnglish (US)
Title of host publicationAdvances in Information and Communication - Proceedings of the 2020 Future of Information and Communication Conference FICC
EditorsKohei Arai, Supriya Kapoor, Rahul Bhatia
PublisherSpringer
Pages419-435
Number of pages17
ISBN (Print)9783030394417
DOIs
StatePublished - Jan 1 2020
EventFuture of Information and Communication Conference, FICC 2020 - San Francisco, United States
Duration: Mar 5 2020Mar 6 2020

Publication series

NameAdvances in Intelligent Systems and Computing
Volume1130 AISC
ISSN (Print)2194-5357
ISSN (Electronic)2194-5365

Conference

ConferenceFuture of Information and Communication Conference, FICC 2020
CountryUnited States
CitySan Francisco
Period3/5/203/6/20

    Fingerprint

Keywords

  • Acceleration
  • Block-circulant matrix
  • Deep learning
  • Parallel computing

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science(all)

Cite this

Pugdeethosapol, K., Jin, Z., Rider, D., & Qiu, Q. (2020). Accelerating Block-Circulant Matrix-Based Neural Network Layer on a General Purpose Computing Platform: A Design Guideline. In K. Arai, S. Kapoor, & R. Bhatia (Eds.), Advances in Information and Communication - Proceedings of the 2020 Future of Information and Communication Conference FICC (pp. 419-435). (Advances in Intelligent Systems and Computing; Vol. 1130 AISC). Springer. https://doi.org/10.1007/978-3-030-39442-4_32