TY - JOUR
T1 - Frequency Propagation
T2 - Multimechanism Learning in Nonlinear Physical Networks
AU - Anisetti, Vidyesh Rao
AU - Kandala, Ananth
AU - Scellier, Benjamin
AU - Schwarz, J. M.
N1 - Publisher Copyright:
© 2024 Massachusetts Institute of Technology.
PY - 2024/3/21
Y1 - 2024/3/21
N2 - We introduce frequency propagation, a learning algorithm for nonlinear physical networks. In a resistive electrical circuit with variable resistors, an activation current is applied at a set of input nodes at one frequency and an error current is applied at a set of output nodes at another fre-quency. The voltage response of the circuit to these boundary currents is the superposition of an activation signal and an error signal whose coefficients can be read in different frequencies of the frequency domain. Each conductance is updated proportionally to the product of the two coefficients. The learning rule is local and proved to perform gradient descent on a loss function. We argue that frequency propagation is an in-stance of a multimechanism learning strategy for physical networks, be it resistive, elastic, or flow networks. Multimechanism learning strategies incorporate at least two physical quantities, potentially governed by in-dependent physical mechanisms, to act as activation and error signals in the training process. Locally available information about these two signals is then used to update the trainable parameters to perform gradient descent. We demonstrate how earlier work implementing learning via chemical signaling in flow networks (Anisetti, Scellier, et al., 2023) also falls under the rubric of multimechanism learning.
AB - We introduce frequency propagation, a learning algorithm for nonlinear physical networks. In a resistive electrical circuit with variable resistors, an activation current is applied at a set of input nodes at one frequency and an error current is applied at a set of output nodes at another fre-quency. The voltage response of the circuit to these boundary currents is the superposition of an activation signal and an error signal whose coefficients can be read in different frequencies of the frequency domain. Each conductance is updated proportionally to the product of the two coefficients. The learning rule is local and proved to perform gradient descent on a loss function. We argue that frequency propagation is an in-stance of a multimechanism learning strategy for physical networks, be it resistive, elastic, or flow networks. Multimechanism learning strategies incorporate at least two physical quantities, potentially governed by in-dependent physical mechanisms, to act as activation and error signals in the training process. Locally available information about these two signals is then used to update the trainable parameters to perform gradient descent. We demonstrate how earlier work implementing learning via chemical signaling in flow networks (Anisetti, Scellier, et al., 2023) also falls under the rubric of multimechanism learning.
UR - http://www.scopus.com/inward/record.url?scp=85189120174&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85189120174&partnerID=8YFLogxK
U2 - 10.1162/neco_a_01648
DO - 10.1162/neco_a_01648
M3 - Article
C2 - 38457749
AN - SCOPUS:85189120174
SN - 0899-7667
VL - 36
SP - 596
EP - 620
JO - Neural Computation
JF - Neural Computation
IS - 4
ER -