Algebraic Geometric Comparison of Probability Distributions

We propose a novel algebraic framework for treating probability distributions represented by their cumulants such as the mean and covariance matrix. As an example, we consider the unsupervised learning problem of finding the subspace on which several probability distributions agree. Instead of minimizing an objective function involving the estimated cumulants, we show that by treating the cumulants as elements of the polynomial ring we can directly solve the problem, at a lower computational cost and with higher accuracy. Moreover, the algebraic viewpoint on probability distributions allows us to invoke the theory of Algebraic Geometry, which we demonstrate in a compact proof for an identifiability criterion.

💡 Research Summary

The paper introduces a novel algebraic‑geometric framework for handling probability distributions by treating their cumulants (mean, covariance, higher‑order moments) as elements of a polynomial ring rather than as fixed numerical constants. The authors focus on the unsupervised learning task of finding a linear subspace onto which several distributions become identical after projection. Traditionally this problem is cast as an optimization: one defines a loss measuring the discrepancy between projected cumulants (e.g., the sum of squared differences of projected covariances) and then minimizes it with respect to the projection vector. Such an approach suffers from non‑convexity, local minima, and dependence on initialization.

Instead, the authors reinterpret the problem algebraically. For each pair of distributions they write the equality of projected second‑order cumulants as a homogeneous quadratic equation in the components of the projection vector (v): (v^{\top}(\Sigma_i-\Sigma_j)v = 0). By introducing variables (X, Y) for the two components of (v), each equation becomes a quadratic polynomial (a_{11}X^2 + (a_{12}+a_{21})XY + a_{22}Y^2). The coefficients of these polynomials are collected into vectors (\mathbf{q}_{ij}) that live in a three‑dimensional coefficient space spanned by the monomials ({X^2, XY, Y^2}).

The key insight is that any linear combination of the original equations is still a valid characterization of the solution set. Under generic conditions there exists a linear combination whose polynomial is divisible by one of the variables, say (Y), yielding a factorisation (Y(\alpha X + \beta Y)=0). Assuming the solution does not lie on the trivial line (Y=0), the factor (\alpha X + \beta Y = 0) directly gives the desired direction (v) (up to scale) as ((-,\beta, \alpha)^\top). Thus the problem reduces to finding a polynomial in the span of the (\mathbf{q}_{ij}) that belongs to the subspace generated by ({XY, Y^2}) (or symmetrically ({XY, X^2})).

In practice the covariance matrices are estimated from finite samples and contain noise, so the exact polynomial equations have no common non‑trivial root. The authors therefore compute a least‑squares approximation of the two‑dimensional subspace spanned by the noisy coefficient vectors using singular value decomposition. They then intersect this approximate subspace with the plane spanned by ({XY, Y^2}) (or the alternative plane) to obtain a polynomial that is approximately divisible by (Y). Dividing out the variable yields a linear factor whose coefficients give an estimate of the projection direction. Multiple such intersections are computed (using both variable choices) and combined to improve robustness.

Beyond the algorithmic contribution, the paper provides an identifiability analysis grounded in algebraic geometry. The set of polynomial equations defines an ideal in the polynomial ring; the dimension of the corresponding algebraic set determines how many independent equations (i.e., how many data sets) are required to isolate a unique subspace. For example, when only second‑order cumulants are used in two dimensions, two covariance matrices are insufficient to guarantee a unique direction—at least three are needed unless the difference of two covariances is indefinite. The authors derive a general criterion linking the number of distributions, the ambient dimension, and the order of cumulants to identifiability.

Experimental results compare the proposed algebraic method with Stationary Subspace Analysis (SSA), a state‑of‑the‑art technique for similar problems. Across synthetic benchmarks with varying noise levels and on real neuro‑imaging data, the algebraic approach achieves higher accuracy, faster convergence, and more stable estimates, especially when higher‑order cumulants are incorporated.

The paper situates its contribution within a broader context of algebraic methods in machine learning, citing works on group‑theoretic kernels, algebraic statistics, and information geometry. It emphasizes that while algebraic structures have appeared in statistical modeling, the explicit use of approximate algebra to manipulate noisy cumulant‑derived polynomials is novel.

In conclusion, by treating cumulants as algebraic objects and solving the resulting polynomial system directly, the authors bypass the pitfalls of non‑convex optimization, obtain provable identifiability conditions, and deliver an efficient algorithm applicable to a wide class of problems where solution sets are defined by polynomial constraints. Future directions include extending the framework to higher‑order cumulants for non‑linear subspace discovery, handling discrete distributions via algebraic statistics, and integrating the method into deep learning pipelines that exploit moment‑based regularization.