A fault-tolerant neural network architecture

Tao Liu, Wujie Wen, Lei Jiang, Yanzhi Wang, Chengmo Yang, Gang Quan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

New DNN accelerators based on emerging technologies, such as resistive random access memory (ReRAM), are gaining increasing research attention given their potential of "in-situ" data processing. Unfortunately, device-level physical limitations that are unique to these technologies may cause weight disturbance in memory and thus compromising the performance and stability of DNN accelerators. In this work, we propose a novel fault-tolerant neural network architecture to mitigate the weight disturbance problem without involving expensive retraining. Specifically, we propose a novel collaborative logistic classifier to enhance the DNN stability by redesigning the binary classifiers augmented from both traditional error correction output code (ECOC) and modern DNN training algorithm. We also develop an optimized variable-length "decodefree" scheme to further boost the accuracy under fewer number of classifiers. Experimental results on cutting-edge DNN models and complex datasets show that the proposed fault-tolerant neural network architecture can effectively rectify the accuracy degradation against weight disturbance for DNN accelerators with low cost, thus allowing for its deployment in a variety of mainstream DNNs.

Original languageEnglish (US)
Title of host publicationProceedings of the 56th Annual Design Automation Conference 2019, DAC 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781450367257
DOIs
StatePublished - Jun 2 2019
Externally publishedYes
Event56th Annual Design Automation Conference, DAC 2019 - Las Vegas, United States
Duration: Jun 2 2019Jun 6 2019

Publication series

NameProceedings - Design Automation Conference
ISSN (Print)0738-100X

Conference

Conference56th Annual Design Automation Conference, DAC 2019
CountryUnited States
CityLas Vegas
Period6/2/196/6/19

    Fingerprint

ASJC Scopus subject areas

  • Computer Science Applications
  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Modeling and Simulation

Cite this

Liu, T., Wen, W., Jiang, L., Wang, Y., Yang, C., & Quan, G. (2019). A fault-tolerant neural network architecture. In Proceedings of the 56th Annual Design Automation Conference 2019, DAC 2019 [a55] (Proceedings - Design Automation Conference). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/3316781.3317742