Compositional memory in attractor neural networks with one-step learning

Gregory P. Davis, Garrett E. Katz, Rodolphe J. Gentili, James A. Reggia

Research output: Contribution to journalArticlepeer-review

Abstract

Compositionality refers to the ability of an intelligent system to construct models out of reusable parts. This is critical for the productivity and generalization of human reasoning, and is considered a necessary ingredient for human-level artificial intelligence. While traditional symbolic methods have proven effective for modeling compositionality, artificial neural networks struggle to learn systematic rules for encoding generalizable structured models. We suggest that this is due in part to short-term memory that is based on persistent maintenance of activity patterns without fast weight changes. We present a recurrent neural network that encodes structured representations as systems of contextually-gated dynamical attractors called attractor graphs. This network implements a functionally compositional working memory that is manipulated using top-down gating and fast local learning. We evaluate this approach with empirical experiments on storage and retrieval of graph-based data structures, as well as an automated hierarchical planning task. Our results demonstrate that compositional structures can be stored in and retrieved from neural working memory without persistent maintenance of multiple activity patterns. Further, memory capacity is improved by the use of a fast store-erase learning rule that permits controlled erasure and mutation of previously learned associations. We conclude that the combination of top-down gating and fast associative learning provides recurrent neural networks with a robust functional mechanism for compositional working memory.

Original languageEnglish (US)
Pages (from-to)78-97
Number of pages20
JournalNeural Networks
Volume138
DOIs
StatePublished - Jun 2021

Keywords

  • Compositionality
  • Itinerant attractor dynamics
  • Multiplicative gating
  • One-step learning
  • Programmable neural networks
  • Working memory

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Compositional memory in attractor neural networks with one-step learning'. Together they form a unique fingerprint.

Cite this