Convolutional Embedding for Edit Distance

Edit-distance-based string similarity search has many applications such as spell correction, data de-duplication, and sequence alignment. However, computing edit distance is known to have high complexity, which makes string similarity search challenging for large datasets. In this paper, we propose a deep learning pipeline (called CNN-ED) that embeds edit distance into Euclidean distance for fast approximate similarity search. A convolutional neural network (CNN) is used to generate fixed-length vector embeddings for a dataset of strings and the loss function is a combination of the triplet loss and the approximation error. To justify our choice of using CNN instead of other structures (e.g., RNN) as the model, theoretical analysis is conducted to show that some basic operations in our CNN model preserve edit distance. Experimental results show that CNN-ED outperforms data-independent CGK embedding and RNN-based GRU embedding in terms of both accuracy and efficiency by a large margin. We also show that string similarity search can be significantly accelerated using CNN-based embeddings, sometimes by orders of magnitude.

💡 Research Summary

This paper, titled “Convolutional Embedding for Edit Distance,” addresses the long-standing challenge of the high computational cost of edit distance, which is a fundamental metric for string similarity search in applications like spell checking and data deduplication. The authors propose a novel deep learning pipeline called CNN-ED, which embeds strings into a fixed-dimensional Euclidean space using a Convolutional Neural Network (CNN), such that the Euclidean distance between embeddings approximates their true edit distance.

The core problem is that calculating edit distance is computationally expensive, making large-scale similarity search inefficient. Existing solutions include data-independent embeddings like CGK (embedding into Hamming distance) and learning-based methods like GRU (using Recurrent Neural Networks). However, CGK suffers from low accuracy, while GRU is slow in training and inference due to its sequential RNN nature and produces high-dimensional embeddings, leading to high memory consumption.

The CNN-ED pipeline innovates in several key aspects. First, it represents an input string as a one-hot matrix over its alphabet. This matrix is processed through a series of one-dimensional convolutional and max-pooling layers, followed by a final linear layer that produces a compact, low-dimensional embedding vector (e.g., 128 dimensions). The authors provide a theoretical justification for using CNN by proving that basic operations like max-pooling preserve an upper bound on the edit distance, a claim not established for RNNs. Remarkably, an untrained, randomly initialized CNN was found to sometimes outperform a fully trained GRU model, underscoring the architectural suitability of CNNs for this task.

The model is trained using a combined loss function. It integrates a triplet loss to preserve the relative ordering of distances (ensuring that closer strings in edit distance are also closer in the embedding space) and an approximation error loss to minimize the absolute difference between Euclidean and edit distances. This dual objective ensures both good ranking accuracy and precise distance estimation.

Extensive experiments on five real-world datasets with varying characteristics demonstrate CNN-ED’s superior performance. It significantly outperforms both CGK and GRU in terms of approximation accuracy, sometimes reducing the error of GRU by half while using an embedding vector two orders of magnitude shorter. In terms of efficiency, CNN-ED achieves speedups of up to 30x in training and 200x in inference compared to GRU. When applied to accelerate string similarity search tasks—specifically similarity join and threshold search—CNN-ED-based solutions dramatically outperform state-of-the-art methods like EmbedJoin and HSsearch, achieving orders-of-magnitude faster query processing while maintaining high recall. The method also shows robustness to hyperparameter changes.

In conclusion, CNN-ED presents a compelling, efficient, and accurate framework for edit distance embedding. By successfully translating a complex discrete metric into a simple continuous vector space distance using a well-justified CNN architecture, it opens a promising path for scaling up string similarity search on large datasets.