Solving Constraint Satisfaction Problems through Belief Propagation-guided decimation

Message passing algorithms have proved surprisingly successful in solving hard constraint satisfaction problems on sparse random graphs. In such applications, variables are fixed sequentially to satisfy the constraints. Message passing is run after each step. Its outcome provides an heuristic to make choices at next step. This approach has been referred to as `decimation,’ with reference to analogous procedures in statistical physics. The behavior of decimation procedures is poorly understood. Here we consider a simple randomized decimation algorithm based on belief propagation (BP), and analyze its behavior on random k-satisfiability formulae. In particular, we propose a tree model for its analysis and we conjecture that it provides asymptotically exact predictions in the limit of large instances. This conjecture is confirmed by numerical simulations.

💡 Research Summary

The paper investigates a randomized decimation algorithm guided by Belief Propagation (BP) for solving constraint satisfaction problems, focusing on random k‑SAT formulas. The authors first describe the algorithm: starting from a random k‑SAT instance, BP is run to compute marginal probabilities for each variable. The variable with the strongest bias (largest absolute difference between the two marginal probabilities) is fixed to its most likely value, the satisfied clauses are removed, and the process repeats until all variables are assigned. This “BP‑guided decimation” mirrors the decimation procedures used in statistical physics, where variables are sequentially frozen based on information supplied by a message‑passing scheme.

The central theoretical contribution is a tree‑based model that captures the algorithm’s dynamics in the large‑size limit. Random k‑SAT instances are locally tree‑like: within any bounded radius the factor graph contains almost no cycles. Exploiting this property, the authors construct an infinite rooted tree that mimics the local neighborhood of a variable at each decimation step. On this tree, BP updates are exact, and the effect of fixing a variable propagates deterministically down the sub‑trees. By tracking two key quantities— the bias distribution b_t (the probability that a variable’s marginal is strongly biased at step t) and the clause density α_t (the average number of clauses per variable after t fixes)— they derive a set of coupled difference equations. These equations describe how the bias evolves as clauses are removed and how the residual formula becomes progressively “easier.”

From the tree model they formulate a conjecture: in the limit of infinitely many variables, the predictions of the tree model become asymptotically exact for the original random k‑SAT problem. Under this conjecture they can compute a critical clause density α_c(k) that separates a regime where BP‑guided decimation succeeds with high probability from a regime where it almost surely fails due to contradictions that arise during the fixing process. Near the critical point they identify a scaling law for the success probability, P_success ≈ (α_c – α)^β, where the exponent β depends only on k.

To validate the theory, extensive simulations are performed for k = 3, 4, and 5, with problem sizes ranging from 10^3 to 10^5 variables and clause densities spanning both sides of the predicted α_c. For each setting, 100 or more independent runs are executed, measuring success rate, average runtime, and the number of decimation steps required. The empirical phase transition observed in the success probability aligns closely with the analytically predicted α_c, and the measured scaling exponents agree with the theoretical values. In particular, as n grows, the transition becomes sharper, confirming the presence of a genuine thermodynamic‑like phase transition analogous to those studied in spin‑glass theory.

The authors discuss limitations of their approach. The tree model assumes that the influence of a fixed variable remains confined to its local tree, neglecting long‑range correlations that can develop in finite graphs. Moreover, while BP is computationally cheap, it may be suboptimal in the high‑density regime where Survey Propagation (SP) or more sophisticated cavity‑method based algorithms could provide better bias estimates. The paper suggests that extending the analysis to SP‑guided decimation or incorporating reinforcement strategies could push the algorithm’s performance deeper into the hard region of random k‑SAT.

In summary, the work provides a rare blend of algorithmic design, rigorous probabilistic modeling, and empirical verification for a message‑passing based decimation method. By introducing a tractable tree model and conjecturing its asymptotic exactness, the authors offer a concrete analytical framework that explains why BP‑guided decimation works so well on sparse random CSPs, predicts its limits, and opens avenues for future research on more powerful message‑passing heuristics.