TP Decoding

Tree pruning' (TP) is an algorithm for probabilistic inference on binary Markov random fields. It has been recently derived by Dror Weitz and used to construct the first fully polynomial approximation scheme for counting independent sets up to the tree uniqueness threshold.’ It can be regarded as a clever method for pruning the belief propagation computation tree, in such a way to exactly account for the effect of loops. In this paper we generalize the original algorithm to make it suitable for decoding linear codes, and discuss various schemes for pruning the computation tree. Further, we present the outcomes of numerical simulations on several linear codes, showing that tree pruning allows to interpolate continuously between belief propagation and maximum a posteriori decoding. Finally, we discuss theoretical implications of the new method.

💡 Research Summary

This paper extends the recently introduced “tree‑pruning” (TP) algorithm—originally devised by Dror Weitz for binary Markov random fields (MRFs)—to the problem of decoding binary linear codes. The authors first observe that decoding can be cast as inference on a generalized MRF (gMRF) by exploiting a duality transformation: each variable node i receives a local potential ψ_i(x_i)=Q(y_i|x_i) derived from the channel likelihood, each check node a carries a trivial potential ψ_a(n_a)=1, and the interaction between variable i and check a is encoded by ψ_{ai}(n_a,x_i)=(-1)^{n_a x_i}. With this construction the posterior distribution μ_y(x) of the codeword given the channel output is proportional to the un‑normalized weight ω(x) of the gMRF, establishing Lemma 1 and allowing the decoding problem to be treated as a marginal computation on a binary MRF, albeit with possibly negative edge weights.

Weitz’s original TP method builds a self‑avoiding walk (SAW) tree rooted at a target variable i. The SAW tree contains every non‑reversing walk that never revisits a vertex, except possibly at its endpoint, and it is a finite sub‑tree of the usual computation tree. For permissive binary MRFs (all potentials non‑negative) the root marginal on the SAW tree coincides exactly with the true marginal on the original graph. However, the gMRF arising from decoding is not permissive: the interaction ψ_{ai} can be negative, which makes the SAW tree’s weights non‑probabilistic and renders the ratio‑based derivations in the original papers ill‑defined (e.g., 0/0).

To overcome this obstacle the authors introduce two key modifications. First, they partition the children D(u) of any SAW node u into equivalence classes D₁(u),…,D_k(u) based on the order in which the walk traverses edges that close a loop. Two children belong to the same class if the corresponding extensions of the walk can be concatenated to form a longer self‑avoiding walk that returns to the parent vertex via a higher‑ordered edge. This grouping captures the combinatorial symmetry of loop closures. Second, they redefine the recursive computation of the root marginal. The usual message‑passing equations (6) and (7) are retained for intra‑class aggregation, but the node‑wise update (6) is replaced by a sum over the equivalence classes, guaranteeing that the resulting “generalized root marginal” is always well‑defined even when some edge weights are negative. In effect, the algorithm performs a BP‑like upward pass on the SAW tree, but with messages that are aggregated over loop‑equivalence groups rather than individual edges.

A major practical issue with TP is the size of the SAW tree, which can be exponential in the number of variables. The original TP analysis relied on a “strong spatial mixing” condition—root marginals become insensitive to boundary conditions beyond a certain depth—to justify truncating the tree at a fixed depth t. In coding theory this condition rarely holds because a codeword’s bits are globally constrained by parity checks; the value of a single bit can be determined by a relatively small neighborhood. Consequently, naive truncation leads to poor performance. The paper proposes two more suitable truncation schemes.

1. Depth‑Limited SAW – The tree is cut at a predetermined depth L. All leaves at depth L are assigned a fixed belief derived from the channel observation (e.g., in the binary erasure channel (BEC) a leaf is forced to 0 or 1 if the corresponding symbol is known, otherwise it remains ambiguous). This yields a polynomial‑size tree whose root marginal approximates the true marginal increasingly well as L grows.

2. Loop‑Cut Truncation – Instead of limiting depth, this method stops any walk as soon as it would create a loop longer than a prescribed length ℓ_max. Short loops (which have the strongest impact on BP’s error) are retained, while longer loops are pruned. This approach directly targets the source of BP’s inaccuracy and keeps the tree size under control.

Both schemes are compatible with the generalized message‑passing rules introduced earlier, and they preserve the exactness of the root marginal on the truncated tree (i.e., the marginal is exact for the modified graph).

The authors validate the proposed TP decoder through extensive simulations. They focus primarily on the binary erasure channel because MAP decoding is trivial (Gaussian elimination) and thus provides a clean benchmark. Nevertheless, the TP decoder is not intended as a practical BEC decoder; its value lies in demonstrating the algorithm’s ability to interpolate between BP and MAP. Results show that for modest truncation depths (e.g., L≈5–10) TP already outperforms standard BP, achieving lower bit‑error rates (BER) across a range of erasure probabilities. As the depth increases, the performance curve approaches the MAP curve smoothly, confirming the “continuous interpolation” claim.

For the binary symmetric channel (BSC) and other memoryless binary‑input channels, similar trends are observed. Increasing the truncation depth or allowing longer loops systematically reduces the BER gap to MAP, albeit at higher computational cost. The paper reports experiments on several LDPC ensembles (regular and irregular) and on short block‑length codes where loops are dense. In all cases TP yields a measurable gain over BP, especially in the error‑floor region where short loops dominate the failure events.

Beyond empirical evidence, the paper discusses theoretical implications. By viewing BP as the limit of TP with zero truncation (i.e., an infinite SAW tree with no loop handling) and MAP as the limit of TP with infinite depth (i.e., the full SAW tree), the authors provide a unified perspective on the spectrum of decoding algorithms. They argue that the depth parameter can be treated as a “complexity knob” that trades off runtime against decoding optimality. Moreover, the equivalence‑class grouping offers a new analytical tool to study how specific loop structures affect BP’s fixed points, potentially leading to refined density‑evolution analyses that incorporate loop corrections.

In summary, the paper makes four principal contributions:

1. Reformulation of decoding as inference on a generalized binary MRF, enabling the application of TP to coding problems.
2. Extension of the SAW‑tree construction to non‑permissive gMRFs via equivalence‑class partitioning and a modified upward‑message recursion that remains well‑defined despite negative edge weights.
3. Introduction of two practical truncation strategies (depth‑limited and loop‑cut) that keep the algorithm’s complexity polynomial while preserving the essential loop‑correction benefits.
4. Comprehensive simulation study demonstrating that TP smoothly interpolates between BP and MAP, delivering superior error performance on both erasure and symmetric channels, particularly in regimes where short loops dominate.

The work opens several avenues for future research: proving rigorous performance bounds for TP at finite depth, designing adaptive truncation policies that automatically select the optimal depth based on channel conditions, and extending the framework to non‑binary alphabets or to channels with memory. By bridging the gap between belief propagation and exact MAP decoding, TP decoding represents a significant step toward practical, near‑optimal decoding algorithms for modern error‑correcting codes.