A fault-tolerant neural network architecture

Tao Liu, Wujie Wen, Lei Jiang, Yanzhi Wang, Chengmo Yang, Gang Quan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

New DNN accelerators based on emerging technologies, such as resistive random access memory (ReRAM), are gaining increasing research attention given their potential of "in-situ" data processing. Unfortunately, device-level physical limitations that are unique to these technologies may cause weight disturbance in memory and thus compromising the performance and stability of DNN accelerators. In this work, we propose a novel fault-tolerant neural network architecture to mitigate the weight disturbance problem without involving expensive retraining. Specifically, we propose a novel collaborative logistic classifier to enhance the DNN stability by redesigning the binary classifiers augmented from both traditional error correction output code (ECOC) and modern DNN training algorithm. We also develop an optimized variable-length "decodefree" scheme to further boost the accuracy under fewer number of classifiers. Experimental results on cutting-edge DNN models and complex datasets show that the proposed fault-tolerant neural network architecture can effectively rectify the accuracy degradation against weight disturbance for DNN accelerators with low cost, thus allowing for its deployment in a variety of mainstream DNNs.

Original languageEnglish (US)
Title of host publicationProceedings of the 56th Annual Design Automation Conference 2019, DAC 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781450367257
DOIs
StatePublished - Jun 2 2019
Externally publishedYes
Event56th Annual Design Automation Conference, DAC 2019 - Las Vegas, United States
Duration: Jun 2 2019Jun 6 2019

Publication series

NameProceedings - Design Automation Conference
ISSN (Print)0738-100X

Conference

Conference56th Annual Design Automation Conference, DAC 2019
CountryUnited States
CityLas Vegas
Period6/2/196/6/19

Fingerprint

Network Architecture
Accelerator
Network architecture
Fault-tolerant
Particle accelerators
Classifiers
Disturbance
Classifier
Neural Networks
Neural networks
Data storage equipment
Training Algorithm
Random Access
Error correction
Error Correction
Logistics
Degradation
Binary
Output
Experimental Results

ASJC Scopus subject areas

  • Computer Science Applications
  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Modeling and Simulation

Cite this

Liu, T., Wen, W., Jiang, L., Wang, Y., Yang, C., & Quan, G. (2019). A fault-tolerant neural network architecture. In Proceedings of the 56th Annual Design Automation Conference 2019, DAC 2019 [a55] (Proceedings - Design Automation Conference). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/3316781.3317742

A fault-tolerant neural network architecture. / Liu, Tao; Wen, Wujie; Jiang, Lei; Wang, Yanzhi; Yang, Chengmo; Quan, Gang.

Proceedings of the 56th Annual Design Automation Conference 2019, DAC 2019. Institute of Electrical and Electronics Engineers Inc., 2019. a55 (Proceedings - Design Automation Conference).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Liu, T, Wen, W, Jiang, L, Wang, Y, Yang, C & Quan, G 2019, A fault-tolerant neural network architecture. in Proceedings of the 56th Annual Design Automation Conference 2019, DAC 2019., a55, Proceedings - Design Automation Conference, Institute of Electrical and Electronics Engineers Inc., 56th Annual Design Automation Conference, DAC 2019, Las Vegas, United States, 6/2/19. https://doi.org/10.1145/3316781.3317742
Liu T, Wen W, Jiang L, Wang Y, Yang C, Quan G. A fault-tolerant neural network architecture. In Proceedings of the 56th Annual Design Automation Conference 2019, DAC 2019. Institute of Electrical and Electronics Engineers Inc. 2019. a55. (Proceedings - Design Automation Conference). https://doi.org/10.1145/3316781.3317742
Liu, Tao ; Wen, Wujie ; Jiang, Lei ; Wang, Yanzhi ; Yang, Chengmo ; Quan, Gang. / A fault-tolerant neural network architecture. Proceedings of the 56th Annual Design Automation Conference 2019, DAC 2019. Institute of Electrical and Electronics Engineers Inc., 2019. (Proceedings - Design Automation Conference).
@inproceedings{d0d1f4a5000f4247bdd2ee61f7c52487,
title = "A fault-tolerant neural network architecture",
abstract = "New DNN accelerators based on emerging technologies, such as resistive random access memory (ReRAM), are gaining increasing research attention given their potential of {"}in-situ{"} data processing. Unfortunately, device-level physical limitations that are unique to these technologies may cause weight disturbance in memory and thus compromising the performance and stability of DNN accelerators. In this work, we propose a novel fault-tolerant neural network architecture to mitigate the weight disturbance problem without involving expensive retraining. Specifically, we propose a novel collaborative logistic classifier to enhance the DNN stability by redesigning the binary classifiers augmented from both traditional error correction output code (ECOC) and modern DNN training algorithm. We also develop an optimized variable-length {"}decodefree{"} scheme to further boost the accuracy under fewer number of classifiers. Experimental results on cutting-edge DNN models and complex datasets show that the proposed fault-tolerant neural network architecture can effectively rectify the accuracy degradation against weight disturbance for DNN accelerators with low cost, thus allowing for its deployment in a variety of mainstream DNNs.",
author = "Tao Liu and Wujie Wen and Lei Jiang and Yanzhi Wang and Chengmo Yang and Gang Quan",
year = "2019",
month = "6",
day = "2",
doi = "10.1145/3316781.3317742",
language = "English (US)",
series = "Proceedings - Design Automation Conference",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "Proceedings of the 56th Annual Design Automation Conference 2019, DAC 2019",

}

TY - GEN

T1 - A fault-tolerant neural network architecture

AU - Liu, Tao

AU - Wen, Wujie

AU - Jiang, Lei

AU - Wang, Yanzhi

AU - Yang, Chengmo

AU - Quan, Gang

PY - 2019/6/2

Y1 - 2019/6/2

N2 - New DNN accelerators based on emerging technologies, such as resistive random access memory (ReRAM), are gaining increasing research attention given their potential of "in-situ" data processing. Unfortunately, device-level physical limitations that are unique to these technologies may cause weight disturbance in memory and thus compromising the performance and stability of DNN accelerators. In this work, we propose a novel fault-tolerant neural network architecture to mitigate the weight disturbance problem without involving expensive retraining. Specifically, we propose a novel collaborative logistic classifier to enhance the DNN stability by redesigning the binary classifiers augmented from both traditional error correction output code (ECOC) and modern DNN training algorithm. We also develop an optimized variable-length "decodefree" scheme to further boost the accuracy under fewer number of classifiers. Experimental results on cutting-edge DNN models and complex datasets show that the proposed fault-tolerant neural network architecture can effectively rectify the accuracy degradation against weight disturbance for DNN accelerators with low cost, thus allowing for its deployment in a variety of mainstream DNNs.

AB - New DNN accelerators based on emerging technologies, such as resistive random access memory (ReRAM), are gaining increasing research attention given their potential of "in-situ" data processing. Unfortunately, device-level physical limitations that are unique to these technologies may cause weight disturbance in memory and thus compromising the performance and stability of DNN accelerators. In this work, we propose a novel fault-tolerant neural network architecture to mitigate the weight disturbance problem without involving expensive retraining. Specifically, we propose a novel collaborative logistic classifier to enhance the DNN stability by redesigning the binary classifiers augmented from both traditional error correction output code (ECOC) and modern DNN training algorithm. We also develop an optimized variable-length "decodefree" scheme to further boost the accuracy under fewer number of classifiers. Experimental results on cutting-edge DNN models and complex datasets show that the proposed fault-tolerant neural network architecture can effectively rectify the accuracy degradation against weight disturbance for DNN accelerators with low cost, thus allowing for its deployment in a variety of mainstream DNNs.

UR - http://www.scopus.com/inward/record.url?scp=85067812171&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067812171&partnerID=8YFLogxK

U2 - 10.1145/3316781.3317742

DO - 10.1145/3316781.3317742

M3 - Conference contribution

T3 - Proceedings - Design Automation Conference

BT - Proceedings of the 56th Annual Design Automation Conference 2019, DAC 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -