A Framework for Robust Lossy Compression of Heavy-Tailed Sources

We study the rate-distortion problem for both scalar and vector memoryless heavy-tailed $α$-stable sources ($0 < α< 2$). Using a recently defined notion of ``strength" as a power measure, we derive the rate-distortion function for $α$-stable sources subject to a constraint on the strength of the error and show it to be logarithmic in the strength-to-distortion ratio. We show how our framework paves the way for finding optimal quantizers for $α$-stable sources and other general heavy-tailed ones. In addition, we study high-rate scalar quantizers and show that uniform ones are asymptotically optimal under the error-strength distortion measure. We compare uniform Gaussian and Cauchy quantizers and show that more representation points for the Cauchy source are required to guarantee the same quantization quality. Our findings generalize the well-known results of rate-distortion and quantization of Gaussian sources ($α= 2$) under a quadratic distortion measure.

💡 Research Summary

The paper tackles the fundamental rate‑distortion problem for memoryless sources whose distributions are heavy‑tailed α‑stable with stability parameter 0 < α < 2. Because such distributions have infinite second moments, the classic mean‑square‑error (MSE) distortion is inappropriate. The authors therefore adopt a recently introduced “strength” (also called α‑power) as the distortion measure. Strength is defined via an infimum over scaling factors that make the relative entropy between the source and a reference α‑stable distribution equal to the entropy of the reference. For α = 2 the strength reduces to the usual standard deviation, and for α = 1 it coincides with a logarithmic expectation characteristic of the Cauchy law.

Using this distortion metric, the authors derive the exact rate‑distortion function for any symmetric α‑stable source. The key insight is that α‑stable distributions maximize differential entropy under a strength constraint, mirroring the well‑known Gaussian‑maximum‑entropy property under a variance constraint. Consequently, the information‑theoretic rate‑distortion function R_I(D) equals the operational rate‑distortion function R(D), and both are given by a simple logarithmic expression:

 R(D) = ½ log₂ (C · D⁻¹),

where the constant C depends only on α and the definition of strength. This formula smoothly interpolates between the Gaussian case (α = 2) and the Cauchy case (α = 1). The result is extended to vector sources, covering both independent‑identically‑distributed (i.i.d.) α‑stable vectors and sub‑Gaussian α‑stable vectors (obtained by scaling a Gaussian vector with a positive α‑stable scalar). In all cases the same logarithmic relationship holds, reflecting the linear scaling of strength with dimension.

The paper then turns to quantization. In the high‑rate regime, the authors analyze scalar quantizers under the strength‑distortion measure. By applying the classic high‑rate analysis (cell‑width Δ versus number of representation points M) and substituting the strength‑based distortion, they show that uniform quantizers are asymptotically optimal. The optimal cell width satisfies Δ ≈ D · M^(−1/α), leading to the same logarithmic rate‑distortion trade‑off derived earlier. A Lagrangian optimization for non‑uniform quantizers yields the same solution, confirming that non‑uniform designs offer no advantage at high rates when strength is the distortion metric.

To illustrate practical implications, the authors compare Cauchy (α = 1) and Gaussian (α = 2) sources. For a fixed distortion level D, the Cauchy source requires roughly twice as many quantization points as the Gaussian source to achieve the same quality, reflecting the heavier tails and higher probability of large deviations. They also present an algorithm for constructing a strength‑optimal quantizer: starting from the strength constraint, they iteratively adjust quantization thresholds to match the target distortion while minimizing the entropy of the quantized output. Applying this algorithm to a Cauchy source in a communication scenario with additive Cauchy noise yields higher achievable transmission rates than using a conventional Gaussian quantizer with the same power budget.

The paper concludes by positioning symmetric α‑stable distributions as the natural benchmark for heavy‑tailed data, analogous to the Gaussian benchmark for finite‑variance data. It highlights that the strength‑based framework provides a unified, mathematically tractable approach to both rate‑distortion theory and quantizer design for heavy‑tailed sources. Future work is suggested on extending the analysis to asymmetric α‑stable laws, dependent multivariate structures, and real‑world applications such as compression of neural network weights, which have been empirically shown to exhibit heavy‑tailed statistics.