Fast k Nearest Neighbor Search using GPU

The recent improvements of graphics processing units (GPU) offer to the computer vision community a powerful processing platform. Indeed, a lot of highly-parallelizable computer vision problems can be significantly accelerated using GPU architecture. Among these algorithms, the k nearest neighbor search (KNN) is a well-known problem linked with many applications such as classification, estimation of statistical properties, etc. The main drawback of this task lies in its computation burden, as it grows polynomially with the data size. In this paper, we show that the use of the NVIDIA CUDA API accelerates the search for the KNN up to a factor of 120.

💡 Research Summary

The paper investigates the acceleration of the k‑Nearest Neighbor (k‑NN) search problem using NVIDIA’s CUDA platform on graphics processing units (GPUs). Recognizing that many computer‑vision algorithms are highly parallelizable, the authors focus on the brute‑force (BF) implementation of k‑NN, which, despite its simplicity, suffers from a computational complexity that grows polynomially with the size of the reference and query sets (O(n · m · d) for distance calculations and O(n · m log m) for sorting). The authors argue that this inherent parallelism makes BF an ideal candidate for GPU execution.

The methodology consists of mapping each query point to a CUDA thread block, where all distances to the reference points are computed in parallel using shared memory to reduce global memory latency. The distance metric can be Euclidean, Manhattan, or any Lp norm. After distance computation, the authors employ either the Thrust library’s parallel sort or a custom bitonic sort kernel to order the distances and select the k smallest values. This design eliminates the need for complex data structures and leverages the massive SIMD capabilities of modern GPUs.

Experimental evaluation compares four implementations: (1) BF in MATLAB via MEX functions (BF‑Matlab), (2) BF in native C (BF‑C), (3) a kd‑tree based approach using the ANN library (KDT‑C), and (4) the proposed BF‑CUDA implementation. Tests are performed on a Pentium 4 3.4 GHz CPU with 2 GB RAM and an NVIDIA GeForce 8800 GTX GPU. The data sets vary in size (N = 1,200 to 38,400 points) and dimensionality (D = 8 to 96). Results, summarized in Table 1, show that BF‑CUDA achieves speed‑ups of up to 120× over BF‑Matlab, 100× over BF‑C, and 40× over the kd‑tree method. For the largest configuration (N = 38,400, D = 96), BF‑Matlab requires roughly one hour, BF‑C about 70 minutes, KDT‑C about 20 minutes, while BF‑CUDA completes the task in only 43 seconds.

A key observation is the insensitivity of BF‑CUDA to the dimensionality of the data. While CPU‑based methods exhibit a linear or super‑linear increase in runtime as D grows, BF‑CUDA’s runtime remains almost constant, with an observed slope of 0.001 in the runtime‑vs‑dimension plot. This behavior is attributed to the fact that distance calculations are fully parallelized on the GPU, and the overhead of data transfer between host and device becomes the dominant factor only for very low dimensions (e.g., D = 8), where the kd‑tree method can be slightly faster because it avoids the transfer step.

The paper also discusses practical implications for applications such as entropy estimation, classification, clustering, and especially content‑based image retrieval (CBIR). In CBIR, local feature descriptors are often limited in size to keep search times reasonable; the GPU‑accelerated BF‑KNN allows larger, more discriminative descriptors without prohibitive computational cost, potentially improving retrieval accuracy.

Limitations are acknowledged. First, the data transfer overhead can dominate for low‑dimensional data, reducing the relative advantage of the GPU approach. Second, the brute‑force algorithm still has O(N²) memory requirements, which may exceed GPU memory for very large data sets. Third, the experiments are confined to an older GPU generation (GeForce 8800 GTX); performance on modern architectures (Pascal, Volta, Ampere) is not evaluated. Finally, the comparison with kd‑tree does not consider approximation errors; while kd‑tree is faster for low dimensions, it may sacrifice accuracy, a trade‑off not quantified in the study.

In conclusion, the authors demonstrate that a straightforward CUDA implementation of brute‑force k‑NN can deliver dramatic speed‑ups—up to two orders of magnitude—over traditional CPU implementations, and it remains robust across a wide range of dimensionalities. This makes GPU‑based k‑NN a viable solution for real‑time or near‑real‑time computer‑vision pipelines that rely on exact nearest‑neighbor queries. Future work is suggested in the areas of memory‑efficient data streaming, exploiting newer GPU features (e.g., unified memory, tensor cores), and integrating approximate nearest‑neighbor schemes to balance accuracy and scalability.