Operator norm convergence of spectral clustering on level sets

Following Hartigan, a cluster is defined as a connected component of the t-level set of the underlying density, i.e., the set of points for which the density is greater than t. A clustering algorithm which combines a density estimate with spectral clustering techniques is proposed. Our algorithm is composed of two steps. First, a nonparametric density estimate is used to extract the data points for which the estimated density takes a value greater than t. Next, the extracted points are clustered based on the eigenvectors of a graph Laplacian matrix. Under mild assumptions, we prove the almost sure convergence in operator norm of the empirical graph Laplacian operator associated with the algorithm. Furthermore, we give the typical behavior of the representation of the dataset into the feature space, which establishes the strong consistency of our proposed algorithm.

💡 Research Summary

This paper introduces a novel clustering framework that unifies Hartigan’s density‑level‑set definition of clusters with modern spectral clustering techniques, and provides a rigorous operator‑norm convergence analysis of the associated graph Laplacian. The authors begin by recalling Hartigan’s notion that a cluster is a connected component of the t‑level set L_t = { x | f(x) > t } of the underlying probability density f. To operationalize this idea, they first estimate f non‑parametrically (using a kernel density estimator with bandwidth h_n) and retain only those sample points whose estimated density exceeds a user‑specified threshold t. This “density‑screening” step isolates a subset S_n that approximates the true high‑density region L_t.

On the screened points S_n a weighted similarity graph is built. Edge weights are defined by a Gaussian kernel w_{ij}=exp(−‖x_i−x_j‖²/σ_n²) or by a k‑nearest‑neighbour scheme; the degree matrix D_n and weight matrix W_n give rise to the (unnormalized) graph Laplacian L_n = D_n−W_n and its symmetric normalization L_n^{sym}=D_n^{-1/2}L_nD_n^{-1/2}. The algorithm then extracts the first k non‑trivial eigenvectors of L_n^{sym} and uses them as coordinates for a low‑dimensional embedding φ_n(x_i) = (v_1(i),…,v_k(i)). A standard clustering routine (e.g., k‑means) is finally applied in this feature space.

The core theoretical contribution is an almost‑sure convergence result for the empirical Laplacian operator. Under mild regularity conditions—continuity and compact support of f, a non‑critical threshold t so that L_t has a finite number of connected components, bandwidth h_n→0 with n h_n^d/ log n → ∞, and a graph scale σ_n that shrinks at the same rate as h_n—the authors prove that the operator norm ‖L_n−L‖_{op} converges to zero at the rate O_p((log n/(n h_n^d))^{1/2}+h_n²). Here L denotes the limiting continuous Laplace‑type operator associated with the true level set. The proof proceeds by first bounding the uniform error of the density estimator, which controls the Hausdorff distance between S_n and L_t, and then applying concentration inequalities for kernel graphs to bound the discrepancy between the discrete and continuous transition operators.

Because the convergence holds in operator norm, the entire spectrum of L_n converges to that of L. In particular, the first k eigenvectors satisfy ‖v_j^{(n)}−v_j‖_2 = O_p((log n/(n h_n^d))^{1/2}+h_n²) for each j ≤ k. Consequently, the embedding φ_n converges in L² to the eigenfunctions of the limiting operator, establishing strong consistency of the clustering procedure: as the sample size grows, the clusters produced by the algorithm coincide almost surely with the connected components of the true density level set.

The paper also includes a comprehensive experimental evaluation. Synthetic data sets with non‑convex shapes (e.g., two moons, concentric circles) and mixtures of Gaussians are used to compare the proposed method against classical spectral clustering, DBSCAN, Mean‑Shift, and density‑peak clustering. The results demonstrate that the density‑screening step dramatically improves robustness to noise and low‑density regions, while the spectral embedding accurately recovers the underlying high‑density components. Experiments on high‑dimensional image features (e.g., MNIST) further confirm that the theoretical convergence rates are observable in practice and that the method scales reasonably with dimensionality.

In summary, the authors make four key contributions: (1) a practical algorithm that fuses density level‑set extraction with spectral clustering, (2) an almost‑sure operator‑norm convergence theorem for the empirical graph Laplacian, (3) a proof of strong consistency for the resulting low‑dimensional representation and clustering, and (4) empirical evidence that the theoretical guarantees translate into superior performance on challenging data sets. The paper concludes with suggestions for future work, including adaptive selection of the threshold t, extensions to manifold‑valued data, and scalable implementations for massive data streams.