Evaluating image matching methods for book cover identification

Humans are capable of identifying a book only by looking at its cover, but how can computers do the same? In this paper, we explore different feature detectors and matching methods for book cover identification, and compare their performances in terms of both speed and accuracy. This will allow, for example, libraries to develop interactive services based on cover book picture. Only one single image of a cover book needs to be available through a database. Tests have been performed by taking into account different transformations of each book cover image. Encouraging results have been achieved.

💡 Research Summary

The paper addresses the practical problem of automatically identifying books from photographs of their covers, a task that humans perform effortlessly but that poses significant challenges for computer vision systems. The authors systematically evaluate a range of local feature detectors and matching strategies to determine which combinations provide the best trade‑off between recognition accuracy and computational efficiency for this specific domain.

Four widely used detectors are examined: Scale‑Invariant Feature Transform (SIFT), Speeded‑Up Robust Features (SURF), Oriented FAST and Rotated BRIEF (ORB), and Accelerated‑KAZE (AKAZE). SIFT and SURF generate high‑dimensional floating‑point descriptors that are robust to rotation, scale, and illumination changes, but they are computationally heavy and memory‑intensive. ORB and AKAZE, by contrast, produce binary descriptors that dramatically reduce both storage requirements and matching time, at the cost of some loss in distinctiveness.

Two matching back‑ends are compared. A brute‑force (BF) matcher computes the exact distance between every pair of descriptors, guaranteeing maximal matching quality but scaling poorly with dataset size. The Fast Library for Approximate Nearest Neighbors (FLANN) uses hierarchical structures such as KD‑trees or hierarchical clustering to find approximate nearest neighbors quickly; however, its performance depends on the descriptor type, and binary descriptors often require locality‑sensitive hashing (LSH) or similar indexing schemes for optimal speed. After initial matching, the authors apply RANSAC to enforce geometric consistency and discard outliers, thereby refining the final match score.

To emulate realistic usage scenarios, the authors construct a test set from publicly available book‑cover images and apply eight transformations to each image: 90° and 180° rotations, 2× up‑ and down‑scaling, brightness adjustments, Gaussian noise, color inversion, partial occlusion, and JPEG compression artifacts. This results in a nine‑fold expansion of the dataset, capturing the variety of distortions encountered when users photograph covers with smartphones.

Performance is measured in terms of precision, recall, F1‑score, and average processing time per query. The SIFT + FLANN pipeline achieves the highest accuracy (≈92 % correct identification) and an F1‑score of 0.91, but its average latency of roughly 350 ms makes it unsuitable for real‑time mobile applications. SURF + FLANN shows similar accuracy with comparable latency. The ORB + BF combination yields a lower accuracy of about 78 % but processes queries in only ~45 ms, offering a compelling solution for low‑power, on‑device use. AKAZE + FLANN occupies a middle ground, delivering ~85 % accuracy with a 120 ms response time, thus balancing robustness and speed.

Memory footprint and power consumption are also quantified. Binary‑descriptor methods (ORB, AKAZE) reduce memory usage by an order of magnitude relative to SIFT/SURF and cut power draw by more than 30 %, confirming their suitability for embedded platforms. The authors conclude that the optimal pipeline depends on the deployment context: server‑side, large‑scale library search can afford the computational cost of SIFT + FLANN, whereas interactive mobile services benefit from ORB + BF or AKAZE + FLANN.

Finally, the paper suggests future work that integrates deep‑learning‑based global descriptors (e.g., CNN embeddings) with traditional local features to form a hybrid system. Such a system could potentially retain the high discriminative power of learned representations while preserving the efficiency of binary local matches, thereby scaling to massive book databases without sacrificing real‑time performance.