Approximate kernel clustering

In the kernel clustering problem we are given a large $n\times n$ positive semi-definite matrix $A=(a_{ij})$ with $\sum_{i,j=1}^na_{ij}=0$ and a small $k\times k$ positive semi-definite matrix $B=(b_{ij})$. The goal is to find a partition $S_1,…,S_k$ of ${1,… n}$ which maximizes the quantity $$ \sum_{i,j=1}^k (\sum_{(i,j)\in S_i\times S_j}a_{ij})b_{ij}. $$ We study the computational complexity of this generic clustering problem which originates in the theory of machine learning. We design a constant factor polynomial time approximation algorithm for this problem, answering a question posed by Song, Smola, Gretton and Borgwardt. In some cases we manage to compute the sharp approximation threshold for this problem assuming the Unique Games Conjecture (UGC). In particular, when $B$ is the $3\times 3$ identity matrix the UGC hardness threshold of this problem is exactly $\frac{16\pi}{27}$. We present and study a geometric conjecture of independent interest which we show would imply that the UGC threshold when $B$ is the $k\times k$ identity matrix is $\frac{8\pi}{9}(1-\frac{1}{k})$ for every $k\ge 3$.

💡 Research Summary

The paper studies the kernel clustering problem, a generic formulation that arises in machine learning when one wishes to compress a large n × n positive semidefinite (PSD) matrix A (with zero row‑sum, i.e., centered) into a small k × k matrix that best matches a given PSD matrix B. Formally, the objective is

 Clust(A | B) = max_{partition S₁,…,S_k} ∑{i,j=1}^k b{ij} · ∑{(p,q)∈S_i×S_j} a{pq}

or equivalently

 max_{σ:{1,…,n}→{1,…,k}} ∑{p,q=1}^n a{pq} b_{σ(p)σ(q)}.

The authors make three major contributions:

1. A constant‑factor polynomial‑time approximation algorithm.
   They formulate a semidefinite programming (SDP) relaxation of the problem. The SDP variables are unit vectors x₁,…,x_n∈S^{n‑1}. After solving the SDP, a rounding step assigns each x_i to one of k fixed unit vectors v₁,…,v_k∈S^{k‑1} by picking the index that maximizes the inner product ⟨x_i, v_j⟩. The analysis shows that the expected objective value after rounding is at least α times the SDP optimum, where

 α = π·(1 − 1/k) for arbitrary PSD B,

 α = (8π/9)·(1 − 1/k) when B is centered (∑{i,j} b{ij}=0) and spherical (diagonal entries equal 1).

The latter case includes the important situation B = I_k (the identity matrix). The proof hinges on a new Grothendieck‑type inequality (Theorem 1.1) that bounds the ratio between the SDP value and the best discrete labeling.

2. UGC‑based hardness results.
   Assuming the Unique Games Conjecture (UGC), the authors prove that no polynomial‑time algorithm can achieve a factor better than the α above. The hardness proof follows the “dictatorship vs. low‑influence” paradigm: they design a positive‑semidefinite quadratic form that measures the sum of squares of level‑1 Fourier coefficients of a Boolean function. For dictator functions the form evaluates to π²/2 (k = 2 case) or to a value derived from a Gaussian moment maximization problem for larger k.

   For k = 3 (B = I₃) they compute the Gaussian optimum C(3) = 9π/8, which yields a hardness factor of (1 − 1/3)·C(3) = 16π/27. This matches the algorithmic guarantee, establishing the exact UGC threshold for the 3‑cluster case.

3. A geometric conjecture linking the hardness factor to a Gaussian partition problem.
   The hardness factor for general k reduces to a continuous optimization: partition ℝ^{k‑1} (under the standard Gaussian measure) into k measurable sets A₁,…,A_k, let z_i = ∫_{A_i} x dγ(x) be the Gaussian moment of each part, and maximize Σ_i ‖z_i‖². Denote the optimum by C(k). The authors prove that C(2)=2π and C(3)=9π/8, and they show that any optimal partition must be a “simplicial conical” one—essentially a product of a cone in a lower‑dimensional subspace with a full Euclidean space.

   They conjecture that for every k≥3 the optimal partition uses only three cones of angle 2π/3 (the “propeller” partition) in a 2‑dimensional subspace, with the remaining dimensions left empty. If this “propeller conjecture” holds, then C(k)=8π/9·(1 − 1/k) for all k, implying that the algorithm’s approximation ratio is optimal under UGC for every k≥3.

The paper also discusses connections to earlier work on the positive‑semidefinite Grothendieck problem, Max‑Cut, and recent generic SDP‑hardness results for constraint satisfaction problems (Raghavendra 2008). It emphasizes that while a generic SDP framework can give optimal UGC‑based ratios, the specialized SDP and rounding presented here yield explicit constants and a clear geometric interpretation.

In summary, the authors provide (i) a practical, constant‑factor SDP‑based algorithm for kernel clustering, (ii) matching UGC‑based hardness results that pinpoint the exact approximation threshold for k=3 and conjecturally for all k≥3, and (iii) a novel link between discrete clustering hardness and a continuous Gaussian partition problem, opening a new line of inquiry in high‑dimensional geometry.