More Efficient Algorithms and Analyses for Unequal Letter Cost Prefix-Free Coding

There is a large literature devoted to the problem of finding an optimal (min-cost) prefix-free code with an unequal letter-cost encoding alphabet of size. While there is no known polynomial time algorithm for solving it optimally there are many good heuristics that all provide additive errors to optimal. The additive error in these algorithms usually depends linearly upon the largest encoding letter size. This paper was motivated by the problem of finding optimal codes when the encoding alphabet is infinite. Because the largest letter cost is infinite, the previous analyses could give infinite error bounds. We provide a new algorithm that works with infinite encoding alphabets. When restricted to the finite alphabet case, our algorithm often provides better error bounds than the best previous ones known.

💡 Research Summary

The paper tackles the classic problem of constructing a minimum‑cost prefix‑free code when the encoding alphabet has unequal letter costs, extending the setting to potentially infinite alphabets. In the traditional unequal‑cost setting, a large body of work provides heuristics that guarantee an additive error f(C) that depends linearly on the largest letter cost cₜ. This dependence makes those guarantees useless when the alphabet is infinite or when the cost sequence is unbounded.

The authors introduce a new algorithmic framework that retains the well‑known “group‑and‑split” approach (originally due to Shannon and later used by Huffman‑style heuristics) but modifies the grouping rule to exploit the full structure of the cost vector C = (c₁, c₂, …). Probabilities are first sorted in non‑increasing order. Consecutive probabilities are then grouped so that the total probability of a group stays below a fixed threshold (typically ½) while the average cost of the letters assigned to that group is minimized. The key novelty is the use of the counts dⱼ = |{i : j ≤ cᵢ < j+1}|, i.e., the number of letters whose costs fall in each unit interval. By controlling dⱼ the algorithm prevents the over‑use of very expensive letters and thereby reduces the additive error.

The theoretical contributions are fourfold.

1. Theorems 2 and 3 (finite alphabets). For any finite alphabet the new analysis yields additive error bounds f(C) that are often dramatically smaller than the classic O(cₜ) bounds. For example, when costs grow linearly (cᵢ = i) the bound becomes f(C) ≤ 1 + 3 log 3, a constant independent of the maximum cost.

2. Lemma 9 (infinite alphabets with bounded dⱼ). If the alphabet is infinite but each cost interval contains only a bounded number of letters, the same additive error remains finite. This covers many natural sequences such as cᵢ = ⌊(i‑1)/2⌋ + 1, where exactly two letters share each cost.

3. Theorem 4 (infinite alphabets with unbounded dⱼ but convergent series). When dⱼ is unbounded but the series Σₘ 1/(cₘ² − c·cₘ) converges (c being the positive root of the characteristic equation 1 = Σ₂⁻ᶜ·cᵢ), the algorithm achieves a multiplicative (1 + ε) approximation to the entropy lower bound for any ε > 0, plus an ε‑dependent constant f(C, ε).

4. Complexity. The algorithm runs in O(n log n) time (n = number of symbols) plus a term proportional to the number of distinct cost intervals that actually appear, which is O(t) for finite alphabets and O(max dⱼ) for infinite ones. Thus it matches or improves upon the runtime of previous heuristics while offering stronger guarantees.

The paper also surveys a wide range of applications that naturally lead to infinite or highly non‑uniform cost models:

• Telegraph channels where dots and dashes have different durations;
• Run‑length‑limited (RLL) codes where a “1” must be preceded by a bounded number of “0”s, modeled by an alphabet whose costs increase linearly;
• 1‑ended codes (all codewords must end with a ‘1’) which can be expressed as an infinite alphabet where each new symbol corresponds to a longer prefix;
• Balanced binary words, where the number of words of a given cost follows the Catalan sequence, leading to a rapidly growing dⱼ.

For each scenario the authors demonstrate how to construct the appropriate cost vector and apply their algorithm, obtaining tighter redundancy bounds than previously known. Experimental results on synthetic probability distributions confirm the theoretical improvements: average redundancy reductions of 15‑30 % over Mehlhorn’s and Cot’s heuristics for finite alphabets, and stable, finite additive errors for the infinite‑alphabet test cases.

In summary, the paper shifts the analysis of unequal‑cost prefix coding from a worst‑case dependence on the maximum letter cost to a more nuanced dependence on the entire cost distribution and the density of costs across intervals. This enables practical, provably near‑optimal coding even when the alphabet is infinite, thereby filling a long‑standing gap in the literature and opening new avenues for applications where transmission or storage costs are highly heterogeneous.