Polar Codes are Optimal for Lossy Source Coding

It was shown by Goblick that in fact linear codes are sufficient to achieve the rate-distortion bound [3], [4,Section 6.2.3].

Trellis based quantizers [5] were perhaps the first “practical” solution to source compression. Their encoding complexity is linear in the blocklength of the code (Viterbi algorithm). For any rate strictly larger than R(D) the gap between the expected distortion and the design distortion D vanishes exponentially in the constraint length. However, the complexity of the encoding algorithm also scales exponentially with the constraint length.

Given the success of sparse graph codes combined with lowcomplexity message-passing algorithms for the channel coding problem, it is interesting to investigate the performance of such a combination for lossy source compression.

As a first question, we can ask if the codes themselves are suitable for the task. In this respect, Matsunaga and Yamamoto [6] showed that if the degrees of a low-density parity-check (LDPC) ensemble are chosen as large as Θ(log(N )), where N is the blocklength, then this ensemble saturates the ratedistortion bound if optimal encoding is employed. Even more promising, Martininian and Wainwright [7] proved that properly chosen MN codes with bounded degrees are sufficient to achieve the rate-distortion bound under optimal encoding. EPFL, School of Computer, & Communication Sciences, Lausanne, CH-1015, Switzerland, {satish.korada, ruediger.urbanke}@epfl.ch. This work was partially supported by the National Competence Center in Research on Mobile Information and Communication Systems (NCCR-MICS), a center supported by the Swiss National Science Foundation under grant number 5005-67322.

Much less is known about the performance of sparse graph codes under message-passing encoding. In [8] the authors consider binary erasure quantization, the source-compression equivalent of the binary erasure channel (BEC) coding problem. They show that LDPC-based quantizers fail if the parity check density is o(log(N )) but that properly constructed lowdensity generator-matrix (LDGM) based quantizers combined with message-passing encoders are optimal. They exploit the close relationship between the channel coding problem and the lossy source compression problem, together with the fact that LDPC codes achieve the capacity of the BEC under messagepassing decoding, to prove the latter claim.

Regular LDGM codes were considered in [9]. Using nonrigorous methods from statistical physics it was shown that these codes approach rate-distortion bound for large degrees. It was empirically shown that these codes have good performance under a variant of belief propagation algorithm (reinforced belief propagation). In [10] the authors consider check-regular LDGM codes and show using non-rigorous methods that these codes approach the rate-distortion bound for large check degree. Moreover, for any rate strictly larger than R(D), the gap between the achieved distortion and D vanishes exponentially in the check degree. They also observe that belief propagation inspired decimation (BID) algorithms do not perform well in this context. In [11], survey propagation inspired decimation (SID) was proposed as an iterative algorithm for finding the solutions of K-SAT (nonlinear constraints) formulae efficiently. Based on this success, the authors in [10] replaced the parity-check nodes with nonlinear constraints, and empirically showed that using SID one can achieve a performance close to the rate-distortion bound.

The construction in [8] suggests that those LDGM codes whose duals (LDPC) are optimized for the binary symmetric channel (BSC) might be good candidates for the lossy compression of a BSS using message-passing encoding. In [12] the authors consider such LDGM codes and empirically show that by using SID one can approach very close to the rate-distortion bound. They also mention that even BID works well but that it is not as good as SID. Recently, in [13] it was experimentally shown that using BID it is possible to approach the rate-distortion bound closely. The key to making basic BP work well in this context is to choose the code properly. This suggests that in fact the more sophisticated algorithms like SID may not even be necessary.

In [14] the authors consider a different approach. They show that for any fixed γ, ǫ > 0 the rate-distortion pair (R(D) + γ, D+ǫ) can be achieved with complexity C 1 (γ)ǫ -C2(γ) N . Of course, the complexity diverges as γ and ǫ are made smaller. The idea there is to concatenate a small code of rate R+γ with expected distortion D + ǫ. The source sequence is then split into blocks of size equal to the code. The concentration with respect to the blocklength imp

…(Full text truncated)…