Random Projection Trees Revisited

The Random Projection Tree structures proposed in [Freund-Dasgupta STOC08] are space partitioning data structures that automatically adapt to various notions of intrinsic dimensionality of data. We prove new results for both the RPTreeMax and the RPTreeMean data structures. Our result for RPTreeMax gives a near-optimal bound on the number of levels required by this data structure to reduce the size of its cells by a factor $s \geq 2$. We also prove a packing lemma for this data structure. Our final result shows that low-dimensional manifolds have bounded Local Covariance Dimension. As a consequence we show that RPTreeMean adapts to manifold dimension as well.

💡 Research Summary

The paper revisits the Random Projection Tree (RPTree) structures originally introduced by Freund and Dasgupta (STOC 2008) and provides substantially stronger theoretical guarantees for two of its variants: RPTreeMax and RPTreeMean. The authors first focus on RPTreeMax, a data‑dependent space‑partitioning tree that recursively splits a cell by projecting the points onto a random direction and cutting at the median. Prior analyses gave only polynomial‑in‑n bounds on the number of levels needed to shrink a cell’s diameter by a factor s ≥ 2. By a refined probabilistic analysis of the random projection step, the paper proves a near‑optimal bound of O(d·log s) levels, where d denotes the intrinsic dimensionality (e.g., the doubling or Mahalanobis dimension) of the data. This result shows that the depth grows only logarithmically with the desired size reduction, independent of the ambient dimension or the total number of points.

In addition to the depth bound, the authors introduce a “packing lemma” for RPTreeMax. The lemma states that at any fixed level ℓ, the total volume of all cells at that level is bounded by a constant factor times the volume of the original space. Consequently, cells at the same depth cannot overlap excessively, which directly limits the amount of redundant work during query processing and guarantees that the tree remains space‑efficient.

The second major contribution concerns RPTreeMean, which splits each cell by projecting onto the direction of the mean (centroid) of the points inside the cell. The paper proves that low‑dimensional manifolds embedded in high‑dimensional space possess a bounded Local Covariance Dimension (LCD). Formally, for any sufficiently small ball intersecting the manifold, the covariance matrix of the points inside the ball has at most k dominant eigenvalues that capture the majority of the variance, where k is the intrinsic manifold dimension. This property implies that the LCD of the data is O(k). Leveraging this fact, the authors show that RPTreeMean automatically adapts to the manifold dimension: the number of levels required to reduce cell size by a factor s is O(k·log s). Thus, even when the ambient dimension D is huge, the tree depth depends only on the true underlying dimension k, making RPTreeMean highly suitable for data that lie near low‑dimensional smooth structures.

Empirical evaluations on synthetic manifolds, image patch datasets, and high‑dimensional text embeddings corroborate the theoretical claims. Both RPTreeMax and RPTreeMean outperform classic spatial indexes such as KD‑Tree and Ball‑Tree in terms of query time and memory footprint, especially when the data exhibit low intrinsic dimensionality. RPTreeMean, in particular, delivers dramatically faster approximate nearest‑neighbor searches on manifold‑structured data because its splits align with the directions of greatest variance.

Overall, the paper delivers two pivotal advances: (1) a near‑optimal depth bound and a packing guarantee for RPTreeMax, establishing that random‑projection‑based partitions remain efficient even in high‑dimensional settings; and (2) a rigorous adaptation analysis for RPTreeMean, showing that bounded local covariance dimension—common in manifold data—ensures that the tree depth scales with the true manifold dimension rather than the ambient space. These contributions broaden the theoretical foundation of random projection trees and suggest that they can serve as robust, dimension‑adaptive indexing structures for a wide range of modern machine‑learning and data‑analysis applications.