Distributed Arithmetic Coding for the Asymmetric Slepian-Wolf problem

Distributed source coding schemes are typically based on the use of channels codes as source codes. In this paper we propose a new paradigm, termed “distributed arithmetic coding”, which exploits the fact that arithmetic codes are good source as well as channel codes. In particular, we propose a distributed binary arithmetic coder for Slepian-Wolf coding with decoder side information, along with a soft joint decoder. The proposed scheme provides several advantages over existing Slepian-Wolf coders, especially its good performance at small block lengths, and the ability to incorporate arbitrary source models in the encoding process, e.g. context-based statistical models. We have compared the performance of distributed arithmetic coding with turbo codes and low-density parity-check codes, and found that the proposed approach has very competitive performance.

💡 Research Summary

This paper introduces a novel paradigm for distributed source coding (DSC) called Distributed Arithmetic Coding (DAC), which directly leverages the properties of arithmetic coding (AC) to solve the Slepian‑Wolf (SW) problem in both asymmetric and symmetric settings. Traditional DSC approaches rely on channel codes (turbo, LDPC, etc.) to generate syndromes or parity bits that act as compressed representations of a source given side information. While such methods can approach the SW bound, they require very large block lengths (often >10⁵ symbols) to achieve near‑capacity performance, and they lack flexibility in modeling non‑stationary sources.

The DAC framework consists of three stages: (1) Modeling – identical to classical AC, where the encoder estimates symbol probabilities p₀ and p₁, possibly using sophisticated context‑based adaptive models; (2) Rate Allocation – a new stage that, given the source probabilities and the conditional entropy H(X|Y), computes a parameter kₓ that determines “expanded” probabilities e p₀ and e p₁ (with e p₀ ≥ p₀, e p₁ ≥ p₁). These expanded probabilities are used to deliberately enlarge the sub‑intervals in the AC process, allowing the encoder to select any target rate not smaller than the entropy of a conventional AC; (3) Coding – the current interval I′ᵢ is subdivided into two partially overlapping sub‑intervals whose lengths are e p₀·|I′ᵢ| and e p₁·|I′ᵢ|. The overlap creates intentional ambiguity: without side information the decoder cannot uniquely identify the correct sub‑interval, but when the side information Y is available, a soft joint decoder can resolve the ambiguity by evaluating the likelihood of each candidate interval conditioned on Y.

In the asymmetric case, only one source (X) is encoded while the other (Y) is assumed to be perfectly known at the decoder. The rate‑allocation stage selects kₓ so that the resulting code length matches the desired conditional rate H(X|Y) plus a small overhead. Experiments on i.i.d. binary sources with block lengths ranging from 256 to 2048 symbols show that DAC consistently outperforms turbo‑code and LDPC‑based SW coders by about 0.1–0.2 bits per sample, especially when the correlation is strong (e.g., P(X≠Y)=0.01). In such highly correlated regimes, conventional channel codes would need extremely high‑rate (99–100 %) codes, which are inefficient and difficult to implement, whereas DAC achieves the same performance simply by adjusting the expanded probabilities.

The symmetric extension allows both sources to be encoded simultaneously. The paper formulates the rate‑allocation problem as a constrained optimization: given a total target rate R = R₁ + R₂, find the pair (k₁, k₂) that minimizes the overall distortion while satisfying the SW region constraints. A Lagrangian approach yields closed‑form expressions for the optimal k’s under the i.i.d. assumption. The decoder jointly processes the two streams, using each side’s information to resolve the overlapping intervals of the other. Simulation results confirm that the symmetric DAC also beats turbo/LDPC baselines across a range of correlation levels and block sizes.

Beyond performance, DAC offers significant practical advantages. Arithmetic coding is already a mature component in standards such as JPEG‑2000, H.264/AVC, and HEVC, with well‑optimized hardware and software implementations. Adding DAC functionality requires only modest modifications: a rate‑allocation module and the ability to generate partially overlapping intervals. Consequently, existing compression pipelines can be upgraded to support DSC without redesigning the entropy coder from scratch, which is valuable for applications like sensor networks, distributed video coding, and satellite image transmission where encoder complexity must remain low.

The authors also discuss limitations and future work. The current analysis assumes i.i.d. binary sources and known source statistics; extending DAC to non‑stationary or multi‑dimensional sources will require more advanced modeling and possibly online adaptation of the expanded probabilities. Moreover, a rigorous information‑theoretic proof that DAC asymptotically achieves the full SW rate region remains an open problem due to the non‑linear nature of AC. Nonetheless, the experimental evidence presented demonstrates that DAC is a competitive, low‑complexity alternative to channel‑code‑based DSC, especially in scenarios with short blocks and strong source correlation.