Relax, no need to round: integrality of clustering formulations

We study exact recovery conditions for convex relaxations of point cloud clustering problems, focusing on two of the most common optimization problems for unsupervised clustering: $k$-means and $k$-median clustering. Motivations for focusing on convex relaxations are: (a) they come with a certificate of optimality, and (b) they are generic tools which are relatively parameter-free, not tailored to specific assumptions over the input. More precisely, we consider the distributional setting where there are $k$ clusters in $\mathbb{R}^m$ and data from each cluster consists of $n$ points sampled from a symmetric distribution within a ball of unit radius. We ask: what is the minimal separation distance between cluster centers needed for convex relaxations to exactly recover these $k$ clusters as the optimal integral solution? For the $k$-median linear programming relaxation we show a tight bound: exact recovery is obtained given arbitrarily small pairwise separation $\epsilon > 0$ between the balls. In other words, the pairwise center separation is $\Delta > 2+\epsilon$. Under the same distributional model, the $k$-means LP relaxation fails to recover such clusters at separation as large as $\Delta = 4$. Yet, if we enforce PSD constraints on the $k$-means LP, we get exact cluster recovery at center separation $\Delta > 2\sqrt2(1+\sqrt{1/m})$. In contrast, common heuristics such as Lloyd’s algorithm (a.k.a. the $k$-means algorithm) can fail to recover clusters in this setting; even with arbitrarily large cluster separation, k-means++ with overseeding by any constant factor fails with high probability at exact cluster recovery. To complement the theoretical analysis, we provide an experimental study of the recovery guarantees for these various methods, and discuss several open problems which these experiments suggest.

💡 Research Summary

This paper investigates when convex relaxations of two classic clustering formulations—k‑median and k‑means—exactly recover the underlying ground‑truth clusters in a simple geometric model. The model consists of k unit‑radius balls in ℝ^m whose centers are separated by a distance Δ; from each ball n/k points are drawn independently and uniformly. The authors ask: what is the minimal Δ that guarantees that solving a convex relaxation yields an integral solution that coincides with the true clustering?

The first main result concerns the standard linear programming (LP) relaxation of k‑median. Building on dual‑certificate techniques and concentration of measure, the authors prove that for any fixed ε>0, if Δ>2+ε and n is sufficiently large, the k‑median LP is integral with high probability. This improves on earlier work that required Δ≈3. Moreover, they show that the classic primal‑dual approximation algorithm for k‑median also recovers the clusters without needing its second “independent‑set” phase.

The second result addresses the LP relaxation of k‑means. By examining the complementary slackness conditions, they demonstrate that even when Δ<4 the LP admits fractional optimal solutions, and thus fails to recover the clusters. This failure persists for any number of clusters, even k=2, showing that the k‑means LP is fundamentally weaker than the k‑median LP in this geometric setting.

To overcome this limitation, the authors study a semidefinite programming (SDP) relaxation of k‑means that adds a positive‑semidefinite (PSD) constraint to the LP. They introduce a deterministic geometric condition called “average separation” and prove that it holds with high probability when Δ>2√2(1+√(1/m)). Consequently, the k‑means SDP exactly recovers the clusters under this milder separation requirement. The bound improves as the dimension m grows, approaching 2√2. The authors conjecture that the true threshold is Δ>2+ε, matching the k‑median result, and suggest that a refined deterministic condition could close the gap.

The paper also provides an extensive experimental study. Simulations confirm that the k‑median LP achieves near‑perfect recovery for Δ just above 2, while the k‑means LP fails sharply around Δ=4. The SDP succeeds consistently for Δ exceeding the theoretical bound and appears to work even at smaller separations, supporting the conjecture. In contrast, popular heuristics such as Lloyd’s algorithm for k‑means and the k‑means++ initialization (even with over‑seeding) frequently miscluster the data, sometimes with exponentially small probability, even when Δ is arbitrarily large. This highlights the practical advantage of convex relaxations: they not only provide optimality certificates but also enjoy provable exact‑recovery guarantees where standard heuristics do not.

Finally, the authors compare their setting to the stochastic block model (SBM) community‑detection problem. Although both involve recovering hidden structure from random data, the SBM deals with graph edges, whereas this work focuses on Euclidean point clouds. The geometric nature of the problem leads to different thresholds and proof techniques. The paper concludes with several open directions, including tightening the SDP threshold to match the conjectured Δ>2+ε, extending the deterministic average‑separation condition, and exploring exact‑recovery guarantees for other convex relaxations or more general data distributions.