Storing cycles in Hopfield-type networks with pseudoinverse learning rule: admissibility and network topology

Cyclic patterns of neuronal activity are ubiquitous in animal nervous systems, and partially responsible for generating and controlling rhythmic movements such as locomotion, respiration, swallowing and so on. Clarifying the role of the network connectivities for generating cyclic patterns is fundamental for understanding the generation of rhythmic movements. In this paper, the storage of binary cycles in neural networks is investigated. We call a cycle $\Sigma$ admissible if a connectivity matrix satisfying the cycle’s transition conditions exists, and construct it using the pseudoinverse learning rule. Our main focus is on the structural features of admissible cycles and corresponding network topology. We show that $\Sigma$ is admissible if and only if its discrete Fourier transform contains exactly $r={rank}(\Sigma)$ nonzero columns. Based on the decomposition of the rows of $\Sigma$ into loops, where a loop is the set of all cyclic permutations of a row, cycles are classified as simple cycles, separable or inseparable composite cycles. Simple cycles contain rows from one loop only, and the network topology is a feedforward chain with feedback to one neuron if the loop-vectors in $\Sigma$ are cyclic permutations of each other. Composite cycles contain rows from at least two disjoint loops, and the neurons corresponding to the rows in $\Sigma$ from the same loop are identified with a cluster. Networks constructed from separable composite cycles decompose into completely isolated clusters. For inseparable composite cycles at least two clusters are connected, and the cluster-connectivity is related to the intersections of the spaces spanned by the loop-vectors of the clusters. Simulations showing successfully retrieved cycles in continuous-time Hopfield-type networks and in networks of spiking neurons are presented.

💡 Research Summary

The paper addresses the fundamental problem of storing and retrieving cyclic binary patterns—referred to as “cycles”—in Hopfield‑type neural networks. A cycle Σ (an N × p matrix of ±1 entries) is declared admissible if there exists a connectivity matrix W that satisfies the transition condition x(t + 1) = sgn(W x(t)) for every state in the cycle. Using the Moore‑Penrose pseudoinverse, the authors construct a candidate weight matrix W = Σ Σ⁺ and then derive a necessary and sufficient condition for admissibility: the discrete Fourier transform (DFT) of Σ must contain exactly r = rank(Σ) non‑zero columns. In other words, the row space of Σ must align perfectly with an r‑dimensional subspace spanned by the complex exponential eigenvectors of the cyclic shift operator. If any frequency component is missing (zero column) or redundant (more than r non‑zero columns), a perfect weight matrix cannot be formed.

To explore the structural implications of this condition, the rows of Σ are decomposed into “loops,” each loop being the set of all cyclic permutations of a single row vector. This decomposition yields three distinct families of cycles:

1. Simple cycles – all rows belong to a single loop. The rows are cyclic shifts of one another, and the resulting weight matrix is a feed‑forward chain (1 → 2 → … → p) with a single feedback connection from neuron p back to neuron 1. This is the most elementary topology, essentially a directed ring with one extra edge.

2. Separable composite cycles – rows are drawn from two or more disjoint loops that have no linear overlap. Each loop defines a cluster of neurons; the weight matrix becomes block‑diagonal, each block identical to the simple‑cycle topology for its cluster. Consequently, the clusters are completely isolated from one another, allowing the network to store multiple independent rhythmic patterns simultaneously.

3. Inseparable composite cycles – at least two loops share a non‑trivial intersection in their spanned subspaces. Here the block structure of W contains off‑diagonal sub‑matrices that encode inter‑cluster connections. The strength and pattern of these connections are directly related to the dimension of the intersection Span(L_i) ∩ Span(L_j). The more overlap, the richer the coupling, which yields composite rhythms where clusters influence each other’s phase and amplitude.

The authors verify these theoretical predictions with extensive simulations. In a continuous‑time Hopfield‑type dynamics, \