Intrinsic dimension estimation of data by principal component analysis

Estimating intrinsic dimensionality of data is a classic problem in pattern recognition and statistics. Principal Component Analysis (PCA) is a powerful tool in discovering dimensionality of data sets with a linear structure; it, however, becomes ineffective when data have a nonlinear structure. In this paper, we propose a new PCA-based method to estimate intrinsic dimension of data with nonlinear structures. Our method works by first finding a minimal cover of the data set, then performing PCA locally on each subset in the cover and finally giving the estimation result by checking up the data variance on all small neighborhood regions. The proposed method utilizes the whole data set to estimate its intrinsic dimension and is convenient for incremental learning. In addition, our new PCA procedure can filter out noise in data and converge to a stable estimation with the neighborhood region size increasing. Experiments on synthetic and real world data sets show effectiveness of the proposed method.

💡 Research Summary

The paper tackles the classic problem of estimating the intrinsic dimensionality (ID) of data, a task that is central to many machine learning pipelines such as dimensionality reduction, clustering, and anomaly detection. While Principal Component Analysis (PCA) is a well‑established tool for uncovering the dimensionality of linearly structured data, it loses its discriminative power when the underlying data manifold is nonlinear. To overcome this limitation, the authors propose a novel PCA‑based framework that combines a global covering strategy with local PCA analyses, followed by a variance‑based validation step to produce a robust estimate of the intrinsic dimension.

The method begins by constructing a minimal cover of the dataset. Given a radius (r), the algorithm builds a set of balls (or hyper‑spherical neighborhoods) such that every data point belongs to at least one ball, while the total number of balls is minimized. Practically, this is achieved by first constructing a k‑nearest‑neighbor (k‑NN) graph and then applying a greedy set‑cover heuristic that selects the most “useful” balls first. Overlap between balls is allowed; overlapping regions are later handled by weighted averaging of local dimension estimates.

For each ball (C_k) in the cover, the algorithm performs local PCA. The mean of the points inside the ball is subtracted, the covariance matrix (\Sigma_k) is computed, and its eigenvalues (\lambda_1 \ge \dots \ge \lambda_D) are sorted. The cumulative variance ratio \