Asymptotic analysis of the Gaussian kernel matrix for partially noisy data in high dimensions

The Gaussian kernel is one of the most important kernels, applicable to many research fields, including scientific computing and data science. In this paper, we present asymptotic analysis of the Gaussian kernel matrix in high dimension under a statistical model of noisy data. The main result is a nice combination of Karoui’s asymptotic analysis with procedures of constrained low rank matrix approximations. More specifically, Karouli clarified an important asymptotic structure of the Gaussian kernel matrix, leading to strong consistency of the eigenvectors, though the eigenvalues are inconsistent. This paper focuses on the above results and presents a consistent estimator with the use of the smallest eigenvalue, whenever the target kernel matrix tends to low rank in the asymptotic regime. Importantly, asymptotic analysis is given under a statistical model representing partial noise. Although a naive estimator is inconsistent, applying an optimization method for low rank approximations with constraints, we overcome the difficulty caused by the inconsistency, resulting in a new estimator with strong consistency in rank deficient cases.

💡 Research Summary

The paper investigates the asymptotic behavior of Gaussian kernel matrices when the data are high‑dimensional and contaminated by partially structured noise. Building on Karoui’s seminal work on non‑symmetric spectral limits of kernel matrices, the authors consider observations (x_i = s_i + \xi_i) where the signal vectors (s_i) satisfy a deterministic inner‑product limit and have bounded per‑coordinate energy, while the noise vectors (\xi_i) are independent, zero‑mean, and satisfy a strong law of large numbers so that (|\xi_i|^2/d \to \bar\sigma^2), (\xi_i^\top \xi_j/d \to 0) (for (i\neq j)), and (s_i^\top \xi_j/d \to 0). The scaling parameter of the Gaussian kernel is assumed to satisfy (c_d/d \to \gamma>0).

Under these assumptions the kernel matrix (K(x){ij}= \exp{-|x_i-x_j|^2/(c_d d)}) can be decomposed as a Hadamard product of a “signal” kernel (K(s)) and a “noise” kernel (K(\xi)). Lemma 1 shows that for off‑diagonal entries (K(\xi){ij}) converges almost surely to (\exp(-2\gamma^{-1}\bar\sigma^2)). Consequently, \