Autonomous Learning with High-Dimensional Computing Architecture Similar to von Neumann's

We model human and animal learning by computing with high-dimensional vectors (H = 10,000 for example). The architecture resembles traditional (von Neumann) computing with numbers, but the instructions refer to vectors and operate on them in superposition. The architecture includes a high-capacity memory for vectors, analogue of the random-access memory (RAM) for numbers. The model’s ability to learn from data reminds us of deep learning, but with an architecture closer to biology. The architecture agrees with an idea from psychology that human memory and learning involve a short-term working memory and a long-term data store. Neuroscience provides us with a model of the long-term memory, namely, the cortex of the cerebellum. With roots in psychology, biology, and traditional computing, a theory of computing with vectors can help us understand how brains compute. Application to learning by robots seems inevitable, but there is likely to be more, including language. Ultimately we want to compute with no more material and energy than used by brains. To that end, we need a mathematical theory that agrees with psychology and biology, and is suitable for nanotechnology. We also need to exercise the theory in large-scale experiments. Computing with vectors is described here in terms familiar to us from traditional computing with numbers.

💡 Research Summary

The paper proposes a novel computing architecture that models human and animal learning by operating on high‑dimensional vectors (typically D ≈ 10 000). While preserving the classic von Neumann layout of a central processor, random‑access memory, and I/O, the architecture replaces numbers with vectors and introduces a vector‑centric instruction set. Three elementary operations—vector addition (majority‑vote or thresholded sum), coordinate‑wise multiplication (Hadamard product, XOR for binary vectors), and permutation (coordinate re‑ordering)—are sufficient to encode, bind, and manipulate complex symbolic structures. A fourth operation, the dot product (cosine, Hamming similarity, etc.), provides a similarity measure for retrieval and classification.

The system distinguishes a working memory (the CPU‑like module where vector algebra is performed) from a long‑term memory (a high‑capacity, content‑addressable store analogous to Sparse Distributed Memory). The long‑term store can be imagined as a physical substrate resembling the cerebellar circuitry (mossy fibers, granule cells, Purkinje cells, climbing fibers), whose three‑dimensional layout mirrors the bit‑plane organization of early magnetic‑core memories. This biological analogy supports the claim that the architecture aligns with both neuroscience (cerebellar long‑term storage) and classic psychological models of short‑term vs. long‑term memory (Atkinson‑Shiffrin, Tulving).

Because the operations are applied coordinate‑wise, each of the D dimensions can be processed in parallel by a simple 1‑bit circuit. This yields massive parallelism and enables true “in‑memory computing”: data need not be moved before being operated on, and the system is robust to individual circuit failures—the high dimensionality provides redundancy. Unlike conventional numeric processors that require high precision per gate, the vector system tolerates low‑precision, binary‑level computation while achieving high overall fidelity through statistical averaging.

A concrete experiment demonstrates the practicality of the approach. The authors built 10 K‑dimensional profile vectors for 21 European Union languages using only character trigrams. Each letter received a random seed vector; trigrams were encoded by permuting and multiplying the three seed vectors; the language profile was the sum of all trigram vectors in a document. Classification was performed by cosine similarity between a test sentence’s profile and the stored language profiles. On 21 000 test sentences, the system achieved 97.3 % accuracy, with most errors confined to the same language family. The entire process ran on a standard laptop in a single pass, completing in under eight minutes, illustrating that high‑dimensional vector operations alone can capture rich statistical regularities without back‑propagation.

Theoretical analysis shows that the three core operations allow representation of traditional data structures (sets, sequences, trees, stacks) in a “holistic” or “holographic” fashion: sets are formed by addition, bindings by multiplication, and order by permutation. Decoding is possible because multiplication is its own inverse (XOR) and permutations are invertible. This symbolic‑vector algebra provides associative, compositional, and recursive capabilities comparable to those of conventional programming languages, but with the added benefit of content‑addressable retrieval and graceful degradation.

In summary, the paper establishes a high‑dimensional vector computing paradigm that (1) mirrors the dual‑memory organization of human cognition, (2) maps naturally onto known cerebellar circuitry, (3) exploits massive parallelism and fault tolerance inherent in high‑dimensional spaces, and (4) demonstrates real‑world utility in language identification. By integrating insights from psychology, neuroscience, and computer engineering, the authors argue that this architecture offers a biologically plausible, energy‑efficient alternative to deep‑learning‑centric models and opens a path toward autonomous learning robots and other cognitive systems.