Efficient LDPC Codes over GF(q) for Lossy Data Compression

In this paper we consider the lossy compression of a binary symmetric source. We present a scheme that provides a low complexity lossy compressor with near optimal empirical performance. The proposed scheme is based on b-reduced ultra-sparse LDPC codes over GF(q). Encoding is performed by the Reinforced Belief Propagation algorithm, a variant of Belief Propagation. The computational complexity at the encoder is O(.n.q.log q), where is the average degree of the check nodes. For our code ensemble, decoding can be performed iteratively following the inverse steps of the leaf removal algorithm. For a sparse parity-check matrix the number of needed operations is O(n).

💡 Research Summary

The paper addresses the problem of lossy compression for a binary symmetric source (BSS) by introducing a novel coding and decoding framework that leverages ultra‑sparse low‑density parity‑check (LDPC) codes defined over a finite field GF(q). The authors first construct a family of “b‑reduced” LDPC codes, where a small integer b controls the removal or merging of certain check‑variable node pairs in the Tanner graph. This reduction increases the girth of the graph and accentuates its sparsity, yielding an average check‑node degree ⟨d⟩ that remains very low (typically 2–3) even for long block lengths.

Encoding is performed with a Reinforced Belief Propagation (RBP) algorithm, which is a modified version of the standard belief propagation (BP) decoder. RBP introduces reinforcement parameters (α, β) that blend the current message with a weighted portion of the previous iteration’s message. This blending stabilizes the iterative process on ultra‑sparse graphs, mitigates oscillations, and accelerates convergence toward a solution that satisfies the parity constraints while minimizing the Hamming distortion with respect to the original source sequence. The per‑iteration computational cost of RBP is O(⟨d⟩·q·log q) because each message update involves operations over the field GF(q) and a logarithmic factor arising from field arithmetic. Consequently, the total encoding complexity scales as O(⟨d⟩·n·q·log q), where n is the block length.

Decoding exploits the inverse of the Leaf Removal (LR) algorithm. In the LR procedure, nodes of degree one are repeatedly eliminated, and their associated parity equations are solved trivially, progressively simplifying the graph. For reconstruction, the algorithm proceeds backward: starting from the compressed bits (which correspond to a subset of check node values), the decoder re‑introduces the removed variables in reverse order, each time solving a single‑variable equation. Because the underlying graph is ultra‑sparse, a large fraction of variables become leaves early, and the entire recovery process requires only O(n) elementary operations, making it extremely fast and memory‑efficient.

The authors evaluate the scheme through extensive simulations on code lengths ranging from 10⁴ to 10⁵ bits, field sizes q = 4, 8, 16, and reduction parameters b = 1–3. They plot rate‑distortion (R‑D) curves and compare them with the Shannon bound, conventional variational coding, and earlier LDPC‑based lossy compressors. The results demonstrate that the proposed method approaches the theoretical limit within a few percent, while achieving an order‑of‑magnitude reduction in encoding time relative to standard BP‑based compressors. Decoding time remains linear in n, confirming the practicality of the approach for real‑time applications.

A key insight is that ultra‑sparsity, when combined with a modest amount of graph reduction (the b‑reduction), preserves enough parity constraints to enforce a high‑quality reconstruction yet keeps the factor graph simple enough for RBP to converge quickly. The reinforcement parameters play a crucial role: proper tuning balances the trade‑off between convergence speed and the risk of getting trapped in sub‑optimal fixed points. The paper also discusses the impact of the field size q; larger q improves compression efficiency because each symbol carries more information, but it also increases per‑iteration arithmetic cost.

While the study focuses on a binary symmetric source, the authors argue that the methodology extends naturally to non‑binary sources or sources with asymmetric statistics, provided an appropriate mapping to GF(q) symbols is defined. They acknowledge that automatic selection of the reinforcement parameters and the optimal b‑reduction level remains an open problem, suggesting future work on adaptive schemes and on applying the framework to channels with memory or to joint source‑channel coding scenarios.

In summary, the paper contributes a low‑complexity, near‑optimal lossy compression system based on b‑reduced ultra‑sparse LDPC codes over GF(q). By marrying reinforced belief propagation for encoding with an inverse leaf‑removal decoder, it achieves O(⟨d⟩·n·q·log q) encoding complexity and O(n) decoding complexity, while delivering empirical performance that closely tracks the Shannon rate‑distortion bound. This combination of theoretical elegance and practical efficiency makes the approach a promising candidate for high‑throughput, low‑latency compression tasks in modern communication and storage systems.