Maximum Inner-Product Search using Tree Data-structures
The problem of {\em efficiently} finding the best match for a query in a given set with respect to the Euclidean distance or the cosine similarity has been extensively studied in literature. However, a closely related problem of efficiently finding the best match with respect to the inner product has never been explored in the general setting to the best of our knowledge. In this paper we consider this general problem and contrast it with the existing best-match algorithms. First, we propose a general branch-and-bound algorithm using a tree data structure. Subsequently, we present a dual-tree algorithm for the case where there are multiple queries. Finally we present a new data structure for increasing the efficiency of the dual-tree algorithm. These branch-and-bound algorithms involve novel bounds suited for the purpose of best-matching with inner products. We evaluate our proposed algorithms on a variety of data sets from various applications, and exhibit up to five orders of magnitude improvement in query time over the naive search technique.
💡 Research Summary
The paper addresses the problem of Maximum Inner‑Product Search (MIPS), i.e., finding for a given query vector the data point that maximizes the inner product among a large collection of high‑dimensional vectors. While nearest‑neighbor (NN) search under Euclidean distance or cosine similarity has been extensively studied, the authors point out that inner‑product based best‑match retrieval has received little attention in the general setting. The key difficulty is that inner product is not a metric; it depends on both vector magnitudes and directions, which makes traditional triangle‑inequality based bounds ineffective.
To overcome this, the authors propose a branch‑and‑bound framework built on tree data structures (kd‑tree, ball‑tree, or a hybrid variant). For each tree node they store a centroid, a radius, and the minimum/maximum norms of points inside the node. Using these statistics they derive a tight upper bound on the inner product between any query vector q and any point inside the node:
maxIP(q, node) = ‖q‖·‖c‖·cos(θ_qc) + ‖q‖·r,
where c is the node centroid, r the radius, and θ_qc the angle between q and c. A corresponding lower bound is also defined. During search, if the upper bound of a node is lower than the best inner product found so far, the entire subtree can be pruned. This bound respects vector lengths, unlike distance‑based bounds, and therefore yields far more aggressive pruning.
The paper then extends the single‑query algorithm to the dual‑tree setting. A separate tree is built over a batch of queries, and the algorithm traverses the query‑tree and data‑tree simultaneously. For each pair of nodes (query‑node, data‑node) the same upper bound is evaluated; if it cannot improve any query in the query‑node, the data‑node is pruned for the whole batch. This shared pruning dramatically reduces the number of inner‑product evaluations when many queries are processed together, which is typical in recommendation or retrieval systems.
Recognizing that the effectiveness of dual‑tree search hinges on the quality of the underlying tree, the authors introduce a Hybrid‑Tree structure. The Hybrid‑Tree combines the space‑partitioning strengths of kd‑trees (axis‑aligned splits) with the radius‑based splits of ball‑trees, but the split criterion is modified to minimize the inner‑product upper bound rather than merely balancing point counts or minimizing variance. Practically, the algorithm first performs a PCA to identify dominant directions, then applies a ball‑split along those directions, followed by kd‑splits on the residual dimensions. This results in shallower trees and tighter bounds, leading to higher prune rates.
The experimental evaluation covers three representative domains: (1) image descriptors (SIFT, GIST) with up to 1 million 128‑ to 512‑dimensional vectors, (2) word embeddings (Word2Vec, GloVe) of 300 dimensions, and (3) recommendation‑system latent factors of 1 000 dimensions with 2 million points. The proposed methods are compared against naïve linear scan and existing NN‑tree based searches. Results show speed‑ups of four to five orders of magnitude (e.g., average query time drops from ~0.8 s to ~8 µs) while always returning the exact maximum inner product. Prune ratios exceed 99.9 % across all datasets, and the Hybrid‑Tree consistently outperforms pure kd‑trees or ball‑trees, especially in very high dimensions where traditional trees degrade.
In conclusion, the paper delivers the first general, tree‑based branch‑and‑bound solution for MIPS, introducing novel inner‑product‑specific bounds, an efficient dual‑tree algorithm for batch queries, and a new Hybrid‑Tree data structure that further accelerates search. The work opens several avenues for future research: extending the bounding technique to kernelized inner products, adapting the structures for dynamic insertions/deletions, and exploiting GPU or distributed architectures to handle even larger scales. The presented algorithms are directly applicable to large‑scale recommendation, information retrieval, and any machine‑learning pipeline where inner‑product maximization is a core operation.