Learning Gaussian Mixtures with Arbitrary Separation

In this paper we present a method for learning the parameters of a mixture of $k$ identical spherical Gaussians in $n$-dimensional space with an arbitrarily small separation between the components. Our algorithm is polynomial in all parameters other than $k$. The algorithm is based on an appropriate grid search over the space of parameters. The theoretical analysis of the algorithm hinges on a reduction of the problem to 1 dimension and showing that two 1-dimensional mixtures whose densities are close in the $L^2$ norm must have similar means and mixing coefficients. To produce such a lower bound for the $L^2$ norm in terms of the distances between the corresponding means, we analyze the behavior of the Fourier transform of a mixture of Gaussians in 1 dimension around the origin, which turns out to be closely related to the properties of the Vandermonde matrix obtained from the component means. Analysis of this matrix together with basic function approximation results allows us to provide a lower bound for the norm of the mixture in the Fourier domain. In recent years much research has been aimed at understanding the computational aspects of learning parameters of Gaussians mixture distributions in high dimension. To the best of our knowledge all existing work on learning parameters of Gaussian mixtures assumes minimum separation between components of the mixture which is an increasing function of either the dimension of the space $n$ or the number of components $k$. In our paper we prove the first result showing that parameters of a $n$-dimensional Gaussian mixture model with arbitrarily small component separation can be learned in time polynomial in $n$.

💡 Research Summary

The paper “Learning Gaussian Mixtures with Arbitrary Separation” tackles a long‑standing limitation in the algorithmic learning of Gaussian mixture models (GMMs): the need for a minimum separation between component means that grows with the ambient dimension n or the number of components k. The authors present a method that can recover the parameters of a mixture of k identical spherical Gaussians in ℝⁿ even when the means are arbitrarily close, i.e., the separation can be vanishingly small. Their algorithm runs in time polynomial in all parameters except k, which is treated as a constant or a small integer.

The technical core proceeds in three stages. First, the high‑dimensional problem is reduced to a one‑dimensional setting by projecting the data onto an arbitrary line. Because all components share the same variance σ², the projected distribution remains a mixture of one‑dimensional spherical Gaussians, fully described by the projected means {μ_i} and mixing weights {w_i}.

Second, the authors establish a quantitative relationship between the L² distance of two one‑dimensional mixtures and the Euclidean distances between their corresponding means and mixing coefficients. They analyze the Fourier transform of a mixture:

\