Optimal Bounds-Only Pruning for Spatial AkNN Joins

We propose a bounds-only pruning test for exact Euclidean AkNN joins on partitioned spatial datasets. Data warehouses commonly partition large tables and store row group statistics for them to accelerate searches and joins, rather than maintaining indexes. AkNN joins can benefit from such statistics by constructing bounds and localizing join evaluations to a few partitions before loading them to build spatial indexes. Existing pruning methods are overly conservative for bounds-only spatial data because they do not fully capture its directional semantics, thereby missing opportunities to skip unneeded partitions at the earliest stages of a join. We propose a three-bound proximity test to determine whether all points within a partition have a closer neighbor in one partition than in another, potentially occluded partition. We show that our algorithm is both optimal and efficient.

💡 Research Summary

The paper addresses the problem of executing exact Euclidean all‑k‑nearest‑neighbor (AkNN) joins on large spatial datasets that are stored in a partitioned, index‑free fashion, as is common in modern data‑warehouse environments using columnar file formats such as Parquet. In such settings, only row‑group statistics (minimum and maximum values per dimension) are available before any data is read, and building temporary indexes for each partition is costly. Consequently, an early‑stage pruning step that can discard entire partitions based solely on their bounding boxes would dramatically reduce I/O and computational overhead.

Existing pruning techniques rely on point‑to‑bound distance functions (MinDist, MaxDist) or their bound‑to‑bound extensions (BMinDist, BMaxDist). These methods collapse the distance relationship between two partitions into a single interval, ignoring directional information. As a result, when a “middle” partition lies between an origin partition O and a farther candidate partition B, the traditional tests cannot guarantee that B can be omitted; they only stop when the farthest candidate’s MaxDist exceeds the nearest candidate’s MinDist, which is often too conservative.

The authors propose a novel three‑bound proximity test that explicitly incorporates a third partition E (the “evaluation” partition) to capture directional semantics. The core theorem—called the All‑Points Proximity Theorem—states that the condition

 ∀ o′ ∈ Corners(O): MaxDist(o′, E) < MinDist(o′, B)

is equivalent to the desired property that every point o in O is strictly closer to any point in E than to any point in B. In other words, if the farthest possible distance from each corner of O to E is still smaller than the nearest possible distance from that corner to B, then the whole AABB O is guaranteed to have its nearest neighbors in E, making B unnecessary for the join.

The proof proceeds in two directions. The reverse direction follows directly from the definitions of MaxDist and MinDist. The forward direction is more involved: the authors observe that an AABB is a convex set, and any interior point can be expressed as an affine combination of its 2R corners (R is the dimensionality). They define a function g(p) = MaxDist(p,E)² − MinDist(p,B)² and show that g is convex by analyzing each dimension separately. Convexity allows the application of Jensen’s inequality, which guarantees that if g is negative at all corners, it is negative everywhere inside the box. Negative g(p) implies MaxDist(p,E) < MinDist(p,B), establishing the required inequality for all points.

Algorithm 1 (AllPointsCloser) implements this test efficiently: it iterates over the 2R corners of O, computes MinDist to B and MaxDist to E, and returns false as soon as any corner violates the inequality; otherwise it returns true. The runtime is O(R), matching the information‑theoretic lower bound for any bounds‑only method.

Beyond correctness, the authors demonstrate that the three‑bound relation defines a strict partial order among partitions. This order can be used to determine an optimal loading sequence: partitions that dominate others (i.e., can prune them) are loaded first, ensuring that the maximum number of partitions are eliminated before any data is read or temporary indexes are built.

The paper also discusses practical considerations. Since only AABBs are required, the method works with any row‑group statistics that provide per‑dimension minima and maxima, including those derived from spatial hashing schemes like H3. It does not depend on minimum bounding rectangles (MBRs) that require tighter fitting; thus it is applicable to a broader class of partitioning strategies.

Although the authors do not present empirical benchmarks, they argue that the theoretical optimality and linear‑time implementation make the approach highly suitable for real‑world workloads where reading and decompressing thousands of rows per partition is expensive. By pruning partitions early, the overall AkNN join can avoid loading large portions of the dataset into memory and constructing temporary spatial indexes, leading to substantial savings in both I/O and CPU time.

In summary, the paper contributes a mathematically rigorous, optimal, and computationally cheap pruning test for spatial AkNN joins that leverages only partition bounds. It fills a gap in the literature where previous bounds‑only methods were directionally blind, and it offers a practical tool for modern data‑warehouse systems that rely on columnar, index‑free storage.