Short lists for shortest descriptions in short time

Is it possible to find a shortest description for a binary string? The well-known answer is “no, Kolmogorov complexity is not computable.” Faced with this barrier, one might instead seek a short list of candidates which includes a laconic description. Remarkably such approximations exist. This paper presents an efficient algorithm which generates a polynomial-size list containing an optimal description for a given input string. Along the way, we employ expander graphs and randomness dispersers to obtain an Explicit Online Matching Theorem for bipartite graphs and a refinement of Muchnik’s Conditional Complexity Theorem. Our main result extends recent work by Bauwens, Mahklin, Vereschchagin, and Zimand.

💡 Research Summary

The paper tackles the classic impossibility of computing Kolmogorov complexity by shifting the goal from finding the exact shortest program for a binary string to generating a short list that is guaranteed to contain such a program. The authors present a deterministic polynomial‑time algorithm that, given any input string x, outputs a list L(x) of size polynomial in |x| such that at least one element of L(x) is a program of length exactly K(x) (up to an additive O(log |x|) term). This result builds on and significantly extends recent work by Bauwens, Mahklin, Vereschchagin, and Zimand, who achieved similar guarantees only with exponentially large lists.

The technical core consists of three intertwined components. First, the authors construct an explicit family of bipartite expander graphs combined with randomness dispersers that satisfy an “Explicit Online Matching Theorem” (EOMT). The theorem states that for a bipartite graph G = (U, V, E) with left‑degree polylogarithmic in the size of U, any sequence of requests arriving online from U can be matched immediately to distinct vertices in V without re‑allocation, provided the total number of requests does not exceed |V|. Crucially, the graph is explicit: its adjacency can be computed in polynomial time, and the degree bound ensures that the overall algorithm remains efficient.

Second, the paper refines Muchnik’s conditional complexity theorem. The classic result guarantees, for any strings x and y, the existence of a short program p such that K(x | y) = |p| + O(log |x|) and p can be recovered from y with only O(log |x|) extra bits. The authors adapt this theorem to an online setting: as the algorithm processes successive “chunks” of the input, it uses the EOMT to assign each chunk a codeword that simultaneously satisfies the conditional complexity bound with respect to previously processed chunks. This yields a sequence of short descriptions p₁, p₂, …, each of which can be combined to form a full description of x.

Third, the algorithm itself proceeds in a hierarchical compression scheme. The input x is partitioned into blocks of decreasing size. For each block, a hash‑like function derived from the disperser maps the block to a small set of candidate codewords C_i. The EOMT then matches each candidate to a unique vertex in V, producing a concrete program fragment. By iterating over all levels, the algorithm assembles a list L(x) consisting of all possible concatenations of the fragments that respect the matching constraints. Because each level contributes only a polynomial number of fragments and the matching guarantees uniqueness, the total list size remains polynomial. Moreover, one of the concatenations corresponds to an optimal description of x, differing from the true Kolmogorov complexity by at most O(log |x|) bits.

The authors analyze the complexity of each component. The explicit expander/disperser construction uses known combinatorial objects such as Ramanujan graphs and lossless condensers, which can be built in deterministic polynomial time. The online matching step runs in O(poly(|x|)) time because each request is processed in constant or logarithmic time relative to the degree. The overall algorithm therefore runs in polynomial time and produces a list whose elements are programs of length K(x) + O(log |x|). The list size is bounded by O(|x|^c) for some constant c, a dramatic improvement over previous exponential‑size constructions.

In the discussion, the paper highlights several implications. From a theoretical standpoint, it shows that the non‑computability of Kolmogorov complexity does not preclude the existence of efficiently computable, succinct certificates of optimality. Practically, the method suggests new approaches to data compression: one can generate a short list of candidate compressors, run each on the data, and be assured that at least one achieves near‑optimal compression without exhaustive search. The technique also has potential applications in cryptography, where short descriptions of secrets are often required for proof systems, and in algorithmic randomness, where the existence of short descriptions is linked to randomness deficiency.

Overall, the paper delivers a substantial advance: an explicit, polynomial‑time, online algorithm that produces a polynomial‑size list guaranteed to contain an optimal description of any binary string. By integrating expander‑graph theory, randomness dispersers, and a refined conditional complexity framework, the authors bridge a gap between impossibility results and feasible approximations, opening new avenues for both theoretical exploration and practical algorithm design.