Matroid Regression

We propose an algebraic combinatorial method for solving large sparse linear systems of equations locally - that is, a method which can compute single evaluations of the signal without computing the whole signal. The method scales only in the sparsity of the system and not in its size, and allows to provide error estimates for any solution method. At the heart of our approach is the so-called regression matroid, a combinatorial object associated to sparsity patterns, which allows to replace inversion of the large matrix with the inversion of a kernel matrix that is constant size. We show that our method provides the best linear unbiased estimator (BLUE) for this setting and the minimum variance unbiased estimator (MVUE) under Gaussian noise assumptions, and furthermore we show that the size of the kernel matrix which is to be inverted can be traded off with accuracy.

💡 Research Summary

The paper introduces a novel combinatorial‑algebraic framework for estimating a single linear functional ⟨w, x⟩ of an unknown signal x from a large, sparse linear system A x = b + ε, without solving for the full vector x. The central construct is the “regression matroid” L(w | A), defined as the elementary quotient of the linear matroid of the rows of A by the target vector w. Within this matroid, minimal dependent subsets of rows are called circuits. Two types of circuits are distinguished: (i) particular regression circuits, which give a minimal representation of w as a linear combination of rows of A, and (ii) general regression circuits, which are minimal dependencies among rows of A that lie in the kernel of A (i.e., w = 0). Each circuit C is associated with a unique circuit vector λ_C that records the coefficients of the corresponding linear relation.

Given the observation vector b = A x + ε with covariance Σ, any particular regression circuit D yields an unbiased estimator γ(D) = λ_Dᵀ b of ⟨w, x⟩. Its variance is λ_Dᵀ Σ λ_D. Consequently, the optimal linear unbiased estimator (BLUE) is obtained by selecting a linear (or affine) combination of particular circuits that minimizes this quadratic form. The space of all such combinations is an affine space spanned by a particular circuit plus the linear span of general circuits. The variance minimization reduces to solving a small quadratic system involving the “circuit kernel matrix” K = Λ Σ Λᵀ, where Λ stacks the circuit vectors of the chosen circuits as rows. Crucially, the dimension of K equals the number of circuits used, not the size N of the original system, so computational cost depends only on the sparsity pattern and the chosen locality parameter.

The authors prove that the resulting estimator is BLUE for arbitrary zero‑mean noise and, under Gaussian noise, it is also the minimum‑variance unbiased estimator (MVUE). They further show that adding more circuits (i.e., enlarging the local neighbourhood) strictly decreases the variance, establishing a monotone accuracy‑complexity trade‑off.

Several concrete settings illustrate the theory. In graph‑based potential measurement, rows of A are edge incidence vectors e_j − e_i; circuits correspond to graph cycles, and the regression space coincides with the first homology of the graph. In rank‑1 matrix completion, after a logarithmic transformation, rows become edge vectors of a bipartite graph, again yielding a graphic regression matroid. Discrete tomography and other inverse problems fit the same pattern, with A’s rows encoding projection measurements. In each case, the method replaces a global pseudo‑inverse of A with the inversion of a small kernel matrix built from a handful of cycles or low‑support dependencies.

Algorithmically, the paper suggests practical ways to find circuits: for graphic matroids, depth‑first search can enumerate short cycles; for general sparse matrices, one can exploit bounded non‑zero patterns (e.g., (1,ℓ)-sparsity) to locate minimal supports. Once a particular circuit and a set of general circuits are identified, the circuit kernel K is assembled, inverted (or solved via Cholesky), and the optimal weights are applied to the observed b to produce the estimate and its variance, all without ever forming or storing the full matrix A.

In summary, “matroid regression” provides a systematic method to perform local, optimal inference on massive sparse linear systems. By translating sparsity into combinatorial circuits and reducing the estimation problem to a constant‑size kernel inversion, the approach achieves BLUE/MVUE guarantees while scaling with the intrinsic combinatorial structure rather than the ambient dimension. This opens new possibilities for applications where only a few linear functionals of the solution are needed—such as region‑specific reconstruction in medical imaging, targeted recommendations in recommender systems, or localized predictions in network analysis—offering substantial computational savings over traditional spectral or iterative solvers.