Quantization Meets Projection: A Happy Marriage for Approximate k-Nearest Neighbor Search

Approximate $k$-nearest neighbor (AKNN) search is a fundamental problem with wide applications. To reduce memory and accelerate search, vector quantization is widely adopted. However, existing quantization methods either rely on codebooks – whose query speed is limited by costly table lookups – or adopt dimension-wise quantization, which maps each vector dimension to a small quantized code for fast search. The latter, however, suffers from a fixed compression ratio because the quantized code length is inherently tied to the original dimensionality. To overcome these limitations, we propose MRQ, a new approach that integrates projection with quantization. The key insight is that, after projection, high-dimensional vectors tend to concentrate most of their information in the leading dimensions. MRQ exploits this property by quantizing only the information-dense projected subspace – whose size is fully user-tunable – thereby decoupling the quantized code length from the original dimensionality. The remaining tail dimensions are captured using lightweight statistical summaries. By doing so, MRQ boosts the query efficiency of existing quantization methods while achieving arbitrary compression ratios enabled by the projection step. Extensive experiments show that MRQ substantially outperforms the state-of-the-art method, achieving up to 3x faster search with only one-third the quantization bits for comparable accuracy.

💡 Research Summary

The paper introduces Minimized Residual Quantization (MRQ), a novel approach that combines dimensionality reduction via projection with flexible quantization to overcome the limitations of existing approximate k‑nearest neighbor (AKNN) methods. Traditional quantization falls into two families: codebook‑based methods (e.g., Product Quantization, Additive Quantization) that offer flexible compression ratios but suffer from slow distance computation due to table lookups, and dimension‑wise methods (e.g., Binary Quantization, Scalar Quantization, RabitQ) that achieve ultra‑fast SIMD‑friendly distance estimates but are constrained to a fixed code length equal to the original dimensionality, preventing arbitrary compression ratios.

The authors observe that after applying an information‑preserving projection such as PCA, high‑dimensional embeddings exhibit a long‑tailed variance distribution: a relatively small number of leading dimensions capture the majority of the data variance, while the remaining “tail” dimensions contain little information. This empirical finding motivates a two‑part representation: a projected part consisting of the top‑d dimensions (user‑tunable) and a residual part comprising the remaining dimensions.

MRQ quantizes the projected part using an extended version of RabitQ that supports arbitrary bit lengths, thereby allowing users to trade off compression against speed without being bound to the original dimensionality. The residual part is not stored explicitly; instead, lightweight statistics such as its norm and variance are kept. These statistics provide error bounds that can be used during distance estimation and re‑ranking.

Distance computation proceeds in three stages: (1) a fast, coarse estimate using only the quantized projected codes together with residual error bounds; (2) a refined estimate where the original projected vectors replace their quantized versions, still leveraging residual statistics; (3) an exact computation that incorporates the full residual vector if higher precision is required. This multi‑stage framework enables aggressive early pruning while guaranteeing that the final result respects a user‑specified recall target.

Implementation integrates MRQ into an inverted‑file (IVF) index with custom memory layouts that improve cache locality and drastically reduce index construction time. The authors evaluate MRQ on several large‑scale datasets, including text embeddings from OpenAI‑1536, image embeddings from the DEEP dataset, and standard benchmarks like SIFT1M. Compared with state‑of‑the‑art baselines—HNSW graph search, PQ/OPQ, and RabitQ—MRQ achieves up to three times faster query latency while using roughly one‑third of the bits for comparable recall (Recall@10 ≈ 0.95). The method also scales well to billions of vectors, demonstrating that the combination of projection‑driven dimensionality reduction and flexible quantization can simultaneously address memory constraints and speed requirements.

In summary, MRQ contributes (1) a principled analysis of variance concentration after projection, (2) a flexible quantization scheme that decouples code length from original dimensionality, (3) a novel multi‑stage distance computation that exploits residual statistics for tight error bounds, and (4) an efficient IVF‑based implementation that validates the approach on real‑world large‑scale workloads. This work opens a practical path for deploying high‑performance AKNN search in memory‑limited environments without sacrificing accuracy.