C-LSTM: Enabling efficient LSTM using structured compression techniques on FPGAs

Shuo Wang, Zhe Li, Caiwen Ding, Bo Yuan, Qinru Qiu, Yanzhi Wang, Yun Liang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

58 Scopus citations

Abstract

Recently, significant accuracy improvement has been achieved for acoustic recognition systems by increasing the model size of Long Short-Term Memory (LSTM) networks. Unfortunately, the ever-increasing size of LSTM model leads to inefficient designs on FPGAs due to the limited on-chip resources. The previous work proposes to use a pruning based compression technique to reduce the model size and thus speedups the inference on FPGAs. However, the random nature of the pruning technique transforms the dense matrices of the model to highly unstructured sparse ones, which leads to unbalanced computation and irregular memory accesses and thus hurts the overall performance and energy efficiency. In contrast, we propose to use a structured compression technique which could not only reduce the LSTM model size but also eliminate the irregularities of computation and memory accesses. This approach employs block-circulant instead of sparse matrices to compress weight matrices and reduces the storage requirement from (k2) to (k). Fast Fourier Transform algorithm is utilized to further accelerate the inference by reducing the computational complexity from (k2) to (klogk). The datapath and activation functions are quantized as 16-bit to improve the resource utilization. More importantly, we propose a comprehensive framework called C-LSTM to automatically optimize and implement a wide range of LSTM variants on FPGAs. According to the experimental results, C-LSTM achieves up to 18.8X and 33.5X gains for performance and energy efficiency compared with the state-of-the-art LSTM implementation under the same experimental setup, and the accuracy degradation is very small.

Original languageEnglish (US)
Title of host publicationFPGA 2018 - Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
PublisherAssociation for Computing Machinery, Inc
Pages11-20
Number of pages10
ISBN (Electronic)9781450356145
DOIs
StatePublished - Feb 15 2018
Event2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, FPGA 2018 - Monterey, United States
Duration: Feb 25 2018Feb 27 2018

Publication series

NameFPGA 2018 - Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
Volume2018-February

Other

Other2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, FPGA 2018
CountryUnited States
CityMonterey
Period2/25/182/27/18

Keywords

  • Block-circulant matrix
  • Compression
  • FFT
  • FPGA
  • LSTM
  • RNNs

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'C-LSTM: Enabling efficient LSTM using structured compression techniques on FPGAs'. Together they form a unique fingerprint.

  • Cite this

    Wang, S., Li, Z., Ding, C., Yuan, B., Qiu, Q., Wang, Y., & Liang, Y. (2018). C-LSTM: Enabling efficient LSTM using structured compression techniques on FPGAs. In FPGA 2018 - Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (pp. 11-20). (FPGA 2018 - Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays; Vol. 2018-February). Association for Computing Machinery, Inc. https://doi.org/10.1145/3174243.3174253