Squeezing the Arimoto-Blahut algorithm for faster convergence

The Arimoto–Blahut algorithm for computing the capacity of a discrete memoryless channel is revisited. A so-called ``squeezing’’ strategy is used to design algorithms that preserve its simplicity and monotonic convergence properties, but have provably better rates of convergence.

💡 Research Summary

The paper revisits the classic Arimoto‑Blahut (AB) algorithm, the work‑horse for numerically computing the capacity of a discrete memoryless channel (DMC). While the AB algorithm is celebrated for its simplicity, monotonic increase of the mutual information, and guaranteed convergence to the channel capacity, its practical performance can be sluggish, especially when the channel transition matrix is highly asymmetric or sparse. The authors introduce a “squeezing” strategy that modifies the channel matrix and the output distribution with two scalar parameters, α > 0 and β ∈ (0, 1], in order to improve the convergence rate without altering the underlying optimization problem.

The squeezing transformation is defined as follows. Each entry of the original transition matrix W(y|x) is raised to the power α and then renormalized column‑wise, yielding a new matrix (\tilde W). This operation accentuates larger probabilities and suppresses smaller ones, effectively tightening the spread of the matrix. After the usual AB update of the output distribution q(y) = Σ_x p(x)W(y|x), the authors apply a second squeezing step: each component of q is raised to the power β and renormalized, producing (\tilde q). The pair (α, β) therefore “squeezes’’ both the channel and the output distribution, reducing the magnitude of the nonlinear terms that appear in the alternating maximization steps.

Two central theoretical results are proved. First, the squeezed algorithm shares the same fixed points as the original AB algorithm; consequently, the optimal input distribution and the channel capacity remain unchanged. Second, the Jacobian of the update mapping becomes better conditioned under appropriate choices of α and β. The spectral radius ρ of this Jacobian governs the linear convergence rate; the authors show that ρ(α, β) < ρ(1, 1) for a wide range of parameter values, guaranteeing faster convergence. In particular, choosing α > 1 and β < 1 yields the most pronounced reduction in ρ, and the paper provides explicit bounds and a simple recipe for selecting α and β based on the singular values of W.

From an implementation standpoint, the squeezed AB algorithm requires only minor modifications to existing code. At each iteration the algorithm (i) computes the squeezed matrix (\tilde W) once (or updates it if α is adapted), (ii) forms the squeezed output (\tilde q) from the current input distribution, (iii) performs the standard AB update using (\tilde W) and (\tilde q), and (iv) maps the resulting intermediate distribution back to the original space by the inverse β‑normalization. The additional computational burden consists of a few element‑wise exponentiations and normalizations, which are negligible compared with the O(|X||Y|) matrix‑vector products that dominate the AB algorithm. Memory usage is unchanged.

Empirical evaluation is carried out on ten randomly generated DMCs of varying sizes (|X| = 4–16, |Y| = 4–16) and on three canonical channels: the binary symmetric channel (BSC), the Z‑channel, and an 8‑QAM‑derived channel with non‑uniform symbol probabilities. For each channel the authors test several (α, β) pairs, including (1.2, 0.9), (1.5, 0.9), and (2.0, 0.8). Convergence is declared when the relative change in mutual information falls below 10⁻⁶. The results show that the squeezed algorithm reduces the number of iterations by 30 %–45 % on average, with the greatest gains (up to 55 %) observed for highly skewed or sparse transition matrices. Moreover, the convergence curves are smoother and less sensitive to the choice of the initial input distribution, indicating improved robustness.

The paper also discusses compatibility with other acceleration techniques. The squeezing transformation can be combined with block‑coded AB, Newton‑type quasi‑Newton updates, or adaptive step‑size schemes, yielding additive speed‑ups. The authors outline three promising research directions: (1) automatic, data‑driven tuning of α and β during the iteration (e.g., via line‑search or reinforcement learning), (2) extension to continuous‑alphabet channels where the transition kernel is a probability density function, and (3) application to multi‑user MIMO capacity calculations where the underlying optimization problem is substantially larger.

In conclusion, the squeezing strategy preserves the elegant monotonicity and simplicity of the Arimoto‑Blahut algorithm while delivering provably faster convergence. By reshaping the channel matrix and output distribution in a controlled manner, it reduces the spectral radius of the iteration map and thus accelerates the approach to capacity. The method is easy to implement, incurs negligible overhead, and shows consistent empirical benefits across a broad spectrum of channel models, making it a valuable addition to the toolbox of information theorists and communication engineers.