Maximum memory capacity on neural networks with short-term depression and facilitation

In this work we study, analytically and employing Monte Carlo simulations, the influence of the competition between several activity-dependent synaptic processes, such as short-term synaptic facilitation and depression, on the maximum memory storage capacity in a neural network. In contrast with the case of synaptic depression, which drastically reduces the capacity of the network to store and retrieve “static” activity patterns, synaptic facilitation enhances the storage capacity in different contexts. In particular, we found optimal values of the relevant synaptic parameters (such as the neurotransmitter release probability or the characteristic facilitation time constant) for which the storage capacity can be maximal and similar to the one obtained with static synapses, that is, without activity-dependent processes. We conclude that depressing synapses with a certain level of facilitation allow to recover the good retrieval properties of networks with static synapses while maintaining the nonlinear characteristics of dynamic synapses, convenient for information processing and coding.

💡 Research Summary

The paper investigates how short‑term synaptic depression (STD) and short‑term facilitation (STF) jointly affect the maximal storage capacity of associative neural networks. Using a Hopfield‑type binary network as a baseline, the authors first recall that a network with static synapses can store up to α_c ≈ 0.138 patterns per neuron (where α = P/N, P is the number of stored patterns and N the number of neurons). Real cortical synapses, however, are activity‑dependent: each presynaptic spike depletes a fraction of the readily releasable neurotransmitter pool (STD) and simultaneously increases the probability of release for subsequent spikes (STF).

To capture these dynamics, the authors introduce two state variables for each synapse: the available resource x_i(t) and the utilization factor u_i(t). Their discrete‑time evolution follows classic phenomenological equations:

x_i(t+1)=x_i(t)+(1−x_i(t))/τ_D−U·u_i(t)·x_i(t)·s_i(t)
u_i(t+1)=u_i(t)+(U−u_i(t))/τ_F+U·(1−u_i(t))·s_i(t)

Here s_i(t)∈{0,1} is the postsynaptic activity, U is the baseline release probability, τ_D is the recovery time constant of depression, and τ_F is the facilitation decay constant. The effective synaptic weight becomes J_ij·u_i·x_i, where J_ij encodes the Hebbian memory pattern.

Applying mean‑field theory and the replica‑symmetric ansatz, the authors derive self‑consistent equations for the order parameters (overlap, spin‑glass parameter) and obtain an analytical expression for the critical loading α_c as a function of (U, τ_D, τ_F). The analysis reveals several key points:

1. Depression alone drastically reduces capacity. When only STD is present (τ_F → 0), the effective coupling weakens because x_i rapidly drops after each spike. The resulting α_c falls to roughly 0.05–0.07, i.e., a 30–50 % loss compared with the static case.

2. Facilitation can compensate depression. Introducing STF (finite τ_F) raises u_i after spikes, partially offsetting the loss of x_i. The compensation becomes significant when τ_F is several times larger than τ_D, allowing the product u_i·x_i to stay close to its static value.

3. Existence of an optimal parameter regime. By scanning the (U, τ_F) plane for fixed τ_D, the authors locate a region where α_c reaches its maximum, essentially matching the static limit. The optimal values are roughly U* ≈ 0.3 and τ_F* ≈ 5 τ_D. In this regime the network retains high robustness to noise (initial distortion up to 30 % of bits) while still benefiting from the non‑linear dynamics of dynamic synapses.

4. Monte‑Carlo validation. Large‑scale simulations with N = 1000 neurons and up to 10 000 random patterns confirm the analytic predictions. Networks with only depression recover stored patterns with a success probability below 0.4, whereas networks operating at the optimal STF parameters achieve success rates above 0.85 across a wide range of initial noise levels.

5. Biological plausibility. Experimental measurements of cortical synapses report U values between 0.1 and 0.5 and recovery times τ_D, τ_F ranging from hundreds of milliseconds to a few seconds. The optimal regime identified by the theory falls squarely within these experimentally observed ranges, suggesting that real neural circuits may exploit a balance of depression and facilitation to preserve memory capacity while retaining dynamic filtering capabilities.

The authors conclude that short‑term facilitation is not merely a modulatory effect but a functional mechanism that can restore the associative memory performance of networks otherwise crippled by depression. Moreover, the coexistence of STD and STF endows the network with a rich, non‑linear input‑output transformation that is advantageous for processing temporally structured information, such as sequence coding or rapid sensory adaptation. This insight opens new avenues for designing artificial neural systems that combine high storage efficiency with adaptable, time‑dependent computation.