The solution space geometry of random linear equations

We consider random systems of linear equations over GF(2) in which every equation binds k variables. We obtain a precise description of the clustering of solutions in such systems. In particular, we prove that with probability that tends to 1 as the number of variables, n, grows: for every pair of solutions \sigma, \tau, either there exists a sequence of solutions \sigma,…,\tau, in which successive elements differ by O(log n) variables, or every sequence of solutions \sigma,…,\tau, contains a step requiring the simultaneous change of \Omega(n) variables. Furthermore, we determine precisely which pairs of solutions are in each category. Our results are tight and highly quantitative in nature. Moreover, our proof highlights the role of unique extendability as the driving force behind the success of Low Density Parity Check codes and our techniques also apply to the problem of so-called pseudo-codewords in such codes.

💡 Research Summary

This paper provides a rigorous, quantitative description of the geometry of the solution space of random systems of linear equations over the field GF(2), where each equation involves exactly k variables (k‑XOR‑SAT). The authors model such a system by a random k‑uniform hypergraph H on n vertices, each vertex representing a variable and each hyperedge an equation. The central object of study is the 2‑core of H, the maximal sub‑hypergraph in which every vertex has degree at least two. The presence or absence of a non‑empty 2‑core determines a sharp phase transition in the connectivity structure of the set of solutions.

When the edge probability p = c n^{‑(k‑1)} is below the critical value c_{k,2} (so that the 2‑core is empty w.h.p.), every variable either has degree 0 or 1. By repeatedly removing degree‑0 or degree‑1 vertices one can reduce the system to an empty set, and any solution can be transformed into any other by a sequence of intermediate solutions that differ in only O(log n) variables at each step. Consequently the whole solution set forms a single, well‑connected component; the diameter of the solution space is Θ(log n) in the sense of Hamming distance.

Above the threshold c_{k,2} a giant 2‑core emerges. Inside the core the authors identify a special combinatorial structure called a “core‑flippable cycle”: a set of vertices each of degree exactly two in the core, arranged so that each belongs to exactly two hyperedges and the edges can be ordered cyclically. Flipping all variables in such a cycle yields another valid core solution because each affected equation contains either zero or two flipped variables. Random hypergraphs contain only a bounded (in expectation) number of such short cycles; w.h.p. the total number of vertices participating in core‑flippable cycles is at most ξ(n) for any slowly growing function ξ.

Using this observation the authors define an equivalence relation on solutions: two solutions are “cycle‑equivalent” if they differ only on variables belonging to core‑flippable cycles (they may differ arbitrarily on variables outside the core). The equivalence classes are called clusters. The main results are:

1. Separation of clusters – For any constant c ≠ c_{k,2} and k ≥ 3, there exists α = α(c,k) > 0 such that w.h.p. any two distinct clusters are α n‑separated, i.e., any pair of solutions from different clusters differ on at least α n variables belonging to the 2‑core.

2. Internal connectivity – There exists Q = Q(c,k) > 0 such that w.h.p. each cluster is Q log n‑connected: for any two solutions in the same cluster there is a path of solutions where consecutive solutions differ in at most Q log n variables.

3. Tightness – The authors show that the O(log n) bound is essentially optimal: there exist pairs of solutions within a cluster that require Ω(log n / log log n) steps, due to the presence of tree‑like components attached to the core.

The proof of internal connectivity is constructive. The authors identify a set B of “free” variables (variables not forced by the core equations). The size of B is linear in n, and each assignment to B uniquely determines a solution within a cluster. By flipping the variables of B one at a time and using the linear structure of the equations, they can adjust the solution while only affecting O(log n) other variables at each step. This yields an efficient algorithm for navigating inside a cluster.

The paper also discusses implications for low‑density parity‑check (LDPC) codes. The 2‑core corresponds to the part of the parity‑check matrix that cannot be peeled away by simple iterative decoding; the notion of “unique extendability”—the ability to extend a partial assignment uniquely to a full codeword—explains why LDPC codes with good peeling properties decode successfully. The techniques developed for analyzing core‑flippable cycles and cluster connectivity translate directly to the study of pseudo‑codewords in LDPC decoding.

In summary, the authors establish that the solution space of random k‑XOR‑SAT undergoes a clean clustering transition exactly when the 2‑core appears. Below the threshold the space is a single, highly connected component; above it the space shatters into exponentially many clusters, each internally well‑connected (logarithmic steps) but mutually separated by a linear Hamming distance. The work provides the first rigorous, quantitative proof of both well‑separation and well‑connectivity for a random CSP, and it introduces powerful combinatorial tools (core‑flippable cycles, peeling processes) that are likely to be useful in the broader study of random constraint satisfaction problems and modern coding theory.