Characterizing and Improving Generalized Belief Propagation Algorithms on the 2D Edwards-Anderson Model

We study the performance of different message passing algorithms in the two dimensional Edwards Anderson model. We show that the standard Belief Propagation (BP) algorithm converges only at high temperature to a paramagnetic solution. Then, we test a Generalized Belief Propagation (GBP) algorithm, derived from a Cluster Variational Method (CVM) at the plaquette level. We compare its performance with BP and with other algorithms derived under the same approximation: Double Loop (DL) and a two-ways message passing algorithm (HAK). The plaquette-CVM approximation improves BP in at least three ways: the quality of the paramagnetic solution at high temperatures, a better estimate (lower) for the critical temperature, and the fact that the GBP message passing algorithm converges also to non paramagnetic solutions. The lack of convergence of the standard GBP message passing algorithm at low temperatures seems to be related to the implementation details and not to the appearance of long range order. In fact, we prove that a gauge invariance of the constrained CVM free energy can be exploited to derive a new message passing algorithm which converges at even lower temperatures. In all its region of convergence this new algorithm is faster than HAK and DL by some orders of magnitude.

💡 Research Summary

This paper investigates the performance of several message‑passing algorithms on the two‑dimensional Edwards‑Anderson (EA) spin‑glass model. The authors first demonstrate that the standard Belief Propagation (BP) algorithm, which corresponds to the Bethe approximation, only converges at relatively high temperatures (β ≲ 0.66, i.e., T ≳ 1.5) and always yields a paramagnetic (zero‑magnetization) fixed point. At lower temperatures BP either diverges or gets trapped in non‑physical solutions, reflecting the well‑known limitation of Bethe‑type approximations on loopy lattices.

To overcome this limitation the authors adopt a Cluster Variational Method (CVM) at the plaquette level, i.e., they treat each elementary square of four spins as a region. By introducing beliefs for plaquettes, links, and spins together with Lagrange multipliers that enforce marginalization constraints, they derive a set of self‑consistent equations that can be written as a Generalized Belief Propagation (GBP) scheme. The messages from a plaquette to a link are parametrized by three fields (U, u_i, u_j) that effectively modify the bare coupling J_ij, while link‑to‑spin messages are single‑field cavity terms u. These equations (Eqs. 5‑6 in the paper) reduce to the ordinary BP cavity equations when the plaquette contributions are set to zero.

Using an average‑case (replica‑symmetric) analysis the authors compute the critical inverse temperature at which the CVM free energy predicts a spin‑glass phase. They obtain β_CVM ≈ 1.22 (T ≈ 0.82), roughly twice the Bethe estimate β_Bethe ≈ 0.66. This indicates that the plaquette‑CVM captures short‑range correlations much better than BP.

When the GBP algorithm is run on single disorder realizations, it indeed converges to a non‑paramagnetic (spin‑glass‑like) fixed point for β ≈ 0.79 (T ≈ 1.27) and ceases to converge around β ≈ 1.0, i.e., before the average‑case transition predicted by the CVM. The authors trace this premature loss of convergence to a gauge freedom inherent in the CVM free energy: the Lagrange multipliers can be transformed without changing the free energy, leading to an under‑determined set of message updates. In the standard implementation this gauge freedom causes oscillations and divergence at low temperature.

To fix the gauge, the authors propose a simultaneous update of all messages together with a constraint that removes the redundant degree of freedom (for example, fixing one of the plaquette‑to‑link U‑fields to zero). The resulting “gauge‑fixed GBP” algorithm converges reliably down to temperatures well below the Bethe limit and even below the average‑case CVM transition. Moreover, its convergence speed is dramatically higher: each full sweep costs O(N) operations, and only a few dozen sweeps are needed, making it 10–100 times faster than the two‑way message‑passing algorithm (HAK) and the Double‑Loop (DL) algorithm, both of which guarantee convergence but are computationally expensive (iteration counts in the hundreds to thousands).

The paper also compares the free‑energy values obtained by the four algorithms (BP, GBP, HAK, DL). In the high‑temperature regime all methods agree, but in the low‑temperature regime the gauge‑fixed GBP and the original GBP achieve lower CVM free energies than HAK and DL, indicating that they find better (more stable) stationary points of the approximated free energy.

In summary, the study shows that: (1) the Bethe/ BP approximation is insufficient for the 2D EA model except at high temperature; (2) a plaquette‑level CVM dramatically improves the quality of the paramagnetic solution and yields a more accurate estimate of the spin‑glass transition; (3) GBP derived from this CVM can converge to non‑paramagnetic solutions, but its low‑temperature stability depends on handling the gauge freedom; (4) fixing the gauge leads to a new GBP algorithm that is both fast (linear‑time per iteration) and robust down to low temperatures, outperforming existing double‑loop and two‑way schemes by orders of magnitude. The results suggest that similar gauge‑fixed GBP approaches could be valuable for more complex lattices (e.g., 3D spin glasses) and for practical applications such as image reconstruction where the underlying graph is planar but contains loops.