Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks

Amar Shrestha, Haowen Fang, Qing Wu, Qinru Qiu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Asynchronous event-driven computation and communication using spikes facilitate the realization of spiking neural networks (SNN) to be massively parallel, extremely energy efficient and highly robust on specialized neuromorphic hardware. However, the lack of a unified robust learning algorithm limits the SNN to shallow networks with low accuracies. Artificial neural networks (ANN), however, have the backpropagation algorithm which can utilize gradient descent to train networks which are locally robust universal function approximators. But backpropagation algorithm is neither biologically plausible nor neuromorphic implementation friendly because it requires: 1) separate backward and forward passes, 2) differentiable neurons, 3) high-precision propagated errors, 4) coherent copy of weight matrices at feedforward weights and the backward pass, and 5) non-local weight update. Thus, we propose an approximation of the backpropagation algorithm completely with spiking neurons and extend it to a local weight update rule which resembles a biologically plausible learning rule spike-timing-dependent plasticity (STDP). This will enable error propagation through spiking neurons for a more biologically plausible and neuromorphic implementation friendly backpropagation algorithm for SNNs. We test the proposed algorithm on various traditional and non-traditional benchmarks with competitive results.

Original languageEnglish (US)
Title of host publicationICONS 2019 - Proceedings of International Conference on Neuromorphic Systems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450376808
DOIs
StatePublished - Jul 23 2019
Event2019 International Conference on Neuromorphic Systems, ICONS 2019 - Knoxville, United States
Duration: Jul 23 2019Jul 25 2019

Publication series

NameACM International Conference Proceeding Series

Conference

Conference2019 International Conference on Neuromorphic Systems, ICONS 2019
CountryUnited States
CityKnoxville
Period7/23/197/25/19

Fingerprint

Backpropagation algorithms
Backpropagation
Neurons
Neural networks
Learning algorithms
Plasticity
Hardware
Communication

Keywords

  • Backpropagation
  • Local Learning
  • Neuromorphic
  • Spike-Timing Dependent Plasticity
  • Spiking Neural Networks

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Shrestha, A., Fang, H., Wu, Q., & Qiu, Q. (2019). Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks. In ICONS 2019 - Proceedings of International Conference on Neuromorphic Systems [a10] (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3354265.3354275

Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks. / Shrestha, Amar; Fang, Haowen; Wu, Qing; Qiu, Qinru.

ICONS 2019 - Proceedings of International Conference on Neuromorphic Systems. Association for Computing Machinery, 2019. a10 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Shrestha, A, Fang, H, Wu, Q & Qiu, Q 2019, Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks. in ICONS 2019 - Proceedings of International Conference on Neuromorphic Systems., a10, ACM International Conference Proceeding Series, Association for Computing Machinery, 2019 International Conference on Neuromorphic Systems, ICONS 2019, Knoxville, United States, 7/23/19. https://doi.org/10.1145/3354265.3354275
Shrestha A, Fang H, Wu Q, Qiu Q. Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks. In ICONS 2019 - Proceedings of International Conference on Neuromorphic Systems. Association for Computing Machinery. 2019. a10. (ACM International Conference Proceeding Series). https://doi.org/10.1145/3354265.3354275
Shrestha, Amar ; Fang, Haowen ; Wu, Qing ; Qiu, Qinru. / Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks. ICONS 2019 - Proceedings of International Conference on Neuromorphic Systems. Association for Computing Machinery, 2019. (ACM International Conference Proceeding Series).
@inproceedings{300b8bfd559d49ff92917996270769d3,
title = "Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks",
abstract = "Asynchronous event-driven computation and communication using spikes facilitate the realization of spiking neural networks (SNN) to be massively parallel, extremely energy efficient and highly robust on specialized neuromorphic hardware. However, the lack of a unified robust learning algorithm limits the SNN to shallow networks with low accuracies. Artificial neural networks (ANN), however, have the backpropagation algorithm which can utilize gradient descent to train networks which are locally robust universal function approximators. But backpropagation algorithm is neither biologically plausible nor neuromorphic implementation friendly because it requires: 1) separate backward and forward passes, 2) differentiable neurons, 3) high-precision propagated errors, 4) coherent copy of weight matrices at feedforward weights and the backward pass, and 5) non-local weight update. Thus, we propose an approximation of the backpropagation algorithm completely with spiking neurons and extend it to a local weight update rule which resembles a biologically plausible learning rule spike-timing-dependent plasticity (STDP). This will enable error propagation through spiking neurons for a more biologically plausible and neuromorphic implementation friendly backpropagation algorithm for SNNs. We test the proposed algorithm on various traditional and non-traditional benchmarks with competitive results.",
keywords = "Backpropagation, Local Learning, Neuromorphic, Spike-Timing Dependent Plasticity, Spiking Neural Networks",
author = "Amar Shrestha and Haowen Fang and Qing Wu and Qinru Qiu",
year = "2019",
month = "7",
day = "23",
doi = "10.1145/3354265.3354275",
language = "English (US)",
series = "ACM International Conference Proceeding Series",
publisher = "Association for Computing Machinery",
booktitle = "ICONS 2019 - Proceedings of International Conference on Neuromorphic Systems",

}

TY - GEN

T1 - Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks

AU - Shrestha, Amar

AU - Fang, Haowen

AU - Wu, Qing

AU - Qiu, Qinru

PY - 2019/7/23

Y1 - 2019/7/23

N2 - Asynchronous event-driven computation and communication using spikes facilitate the realization of spiking neural networks (SNN) to be massively parallel, extremely energy efficient and highly robust on specialized neuromorphic hardware. However, the lack of a unified robust learning algorithm limits the SNN to shallow networks with low accuracies. Artificial neural networks (ANN), however, have the backpropagation algorithm which can utilize gradient descent to train networks which are locally robust universal function approximators. But backpropagation algorithm is neither biologically plausible nor neuromorphic implementation friendly because it requires: 1) separate backward and forward passes, 2) differentiable neurons, 3) high-precision propagated errors, 4) coherent copy of weight matrices at feedforward weights and the backward pass, and 5) non-local weight update. Thus, we propose an approximation of the backpropagation algorithm completely with spiking neurons and extend it to a local weight update rule which resembles a biologically plausible learning rule spike-timing-dependent plasticity (STDP). This will enable error propagation through spiking neurons for a more biologically plausible and neuromorphic implementation friendly backpropagation algorithm for SNNs. We test the proposed algorithm on various traditional and non-traditional benchmarks with competitive results.

AB - Asynchronous event-driven computation and communication using spikes facilitate the realization of spiking neural networks (SNN) to be massively parallel, extremely energy efficient and highly robust on specialized neuromorphic hardware. However, the lack of a unified robust learning algorithm limits the SNN to shallow networks with low accuracies. Artificial neural networks (ANN), however, have the backpropagation algorithm which can utilize gradient descent to train networks which are locally robust universal function approximators. But backpropagation algorithm is neither biologically plausible nor neuromorphic implementation friendly because it requires: 1) separate backward and forward passes, 2) differentiable neurons, 3) high-precision propagated errors, 4) coherent copy of weight matrices at feedforward weights and the backward pass, and 5) non-local weight update. Thus, we propose an approximation of the backpropagation algorithm completely with spiking neurons and extend it to a local weight update rule which resembles a biologically plausible learning rule spike-timing-dependent plasticity (STDP). This will enable error propagation through spiking neurons for a more biologically plausible and neuromorphic implementation friendly backpropagation algorithm for SNNs. We test the proposed algorithm on various traditional and non-traditional benchmarks with competitive results.

KW - Backpropagation

KW - Local Learning

KW - Neuromorphic

KW - Spike-Timing Dependent Plasticity

KW - Spiking Neural Networks

UR - http://www.scopus.com/inward/record.url?scp=85073199496&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073199496&partnerID=8YFLogxK

U2 - 10.1145/3354265.3354275

DO - 10.1145/3354265.3354275

M3 - Conference contribution

AN - SCOPUS:85073199496

T3 - ACM International Conference Proceeding Series

BT - ICONS 2019 - Proceedings of International Conference on Neuromorphic Systems

PB - Association for Computing Machinery

ER -