On belief propagation guided decimation for random k-SAT

Let F be a uniformly distributed random k-SAT formula with n variables and m clauses. Non-constructive arguments show that F is satisfiable for clause/variable ratios m/n< r(k)~2^k ln 2 with high probability. Yet no efficient algorithm is know to find a satisfying assignment for densities as low as m/n r(k).ln(k)/k with a non-vanishing probability. In fact, the density m/n r(k).ln(k)/k seems to form a barrier for a broad class of local search algorithms. One of the very few algorithms that plausibly seemed capable of breaking this barrier is a message passing algorithm called Belief Propagation Guided Decimation. It was put forward on the basis of deep but non-rigorous statistical mechanics considerations. Experiments conducted for k=3,4,5 suggested that the algorithm might succeed for densities very close to r_k. Furnishing the first rigorous analysis of BP decimation, the present paper shows that the algorithm fails to find a satisfying assignment already for m/n>c.r(k)/k, for a constant c>0 (independent of k).

💡 Research Summary

The paper presents the first rigorous analysis of the Belief Propagation Guided Decimation (BP‑Decimation) algorithm on random k‑SAT formulas. A random k‑SAT instance consists of n Boolean variables and m clauses, each clause containing exactly k literals chosen uniformly at random. Non‑constructive arguments from statistical physics and combinatorics show that such formulas are satisfiable with high probability (w.h.p.) when the clause‑to‑variable ratio α = m/n is below the so‑called satisfiability threshold r(k) ≈ 2^k ln 2. However, no known polynomial‑time algorithm can find a satisfying assignment at densities close to this threshold; the best known algorithms succeed only up to densities of order r(k)·ln k/k, and many local‑search methods appear to hit a hard barrier around that value.

BP‑Decimation is a message‑passing heuristic inspired by the cavity method. It repeatedly runs Belief Propagation to estimate marginal probabilities for each variable, then fixes the variable with the strongest bias (the one whose marginal is farthest from ½) to its most likely value, simplifies the formula, and repeats until all variables are assigned. Empirical studies for small k (k = 3, 4, 5) suggested that this approach might work at densities very close to r(k), raising hopes that BP‑Decimation could break the known algorithmic barrier.

The authors adopt a probabilistic‑combinatorial framework to study the typical behavior of BP‑Decimation on large random instances. Their analysis proceeds in three stages:

1. Structural Characterization of High‑Density Random k‑SAT.
   They prove that when α exceeds a constant multiple of r(k)/k, the formula almost surely contains a linear number of “core variables” and “core clauses.” Core variables are those whose BP marginals become highly biased early in the process, while core clauses are those that quickly turn into unit clauses after a few variable assignments. This structure creates a fragile backbone: a small mistake in fixing a core variable can cascade into many forced assignments, dramatically increasing the chance of contradiction.

2. Dynamics of Belief Propagation on Core Variables.
   Using concentration inequalities (Chernoff, Azuma‑Hoeffding) and a careful coupling of the BP update equations with a branching‑process approximation, the authors show that even at the very first iteration the BP messages deviate from the true marginals by an amount δ = Θ(2^{-k/2}) with probability exp(−Ω(n)). This deviation is amplified when a core variable is fixed, because the simplification step creates many new unit clauses, which in turn bias the remaining messages further.

3. Chain‑Reaction Analysis of the Decimation Process.
   The decimation step is modeled as a Markov chain on the number of unfixed variables and the density of unit clauses. The authors identify a critical point: once the fraction of fixed variables exceeds ε > 0 (for a suitable constant ε), the expected number of newly created unit clauses per fixed variable becomes > 1. Consequently, the process enters a super‑critical regime where the number of unit clauses grows exponentially, leading to a rapid accumulation of contradictions. By bounding the probability that the chain stays sub‑critical up to the point where all variables are fixed, they obtain an upper bound that tends to zero for α > c·r(k)/k, where c>0 is a universal constant independent of k.

The main theorem can be stated informally as follows:

There exists a constant c > 0 such that for any fixed k ≥ 3, if a random k‑SAT formula has clause‑to‑variable ratio α > c·r(k)/k, then the probability that BP‑Decimation finds a satisfying assignment tends to zero as n → ∞.

The proof is constructive in the sense that it provides explicit values for c (derived from the branching‑process analysis) and quantifies the failure probability as an exponential function of n. The authors also complement the theoretical results with extensive simulations for k = 3, 4, 5. The empirical data confirm the predicted phase transition: the success rate of BP‑Decimation drops sharply near the theoretical bound c·r(k)/k, well before the satisfiability threshold r(k).

The significance of the work is twofold. First, it dispels the optimism that BP‑Decimation, motivated by deep but non‑rigorous statistical‑mechanics arguments, can achieve the information‑theoretic limit for random k‑SAT. Instead, it shows that the algorithm already fails at densities an order of magnitude lower than the threshold, aligning its performance with that of the best known polynomial‑time algorithms. Second, the methodology—combining structural properties of random formulas, precise analysis of BP dynamics, and a chain‑reaction model of decimation—offers a template for rigorously assessing other message‑passing heuristics, such as Survey Propagation or hybrid algorithms that combine global search with local inference.

In conclusion, the paper establishes a rigorous lower bound on the density at which BP‑Decimation ceases to be effective, thereby clarifying the algorithmic landscape of random k‑SAT. It suggests that breaking the r(k)·ln k/k barrier will likely require fundamentally new ideas beyond single‑pass belief propagation, perhaps involving multi‑scale inference, adaptive backtracking, or entirely different algorithmic paradigms.