Fast low-level pattern matching algorithm

This paper focuses on pattern matching in the DNA sequence. It was inspired by a previously reported method that proposes encoding both pattern and sequence using prime numbers. Although fast, the method is limited to rather small pattern lengths, due to computing precision problem. Our approach successfully deals with large patterns, due to our implementation that uses modular arithmetic. In order to get the results very fast, the code was adapted for multithreading and parallel implementations. The method is reduced to assembly language level instructions, thus the final result shows significant time and memory savings compared to the reference algorithm.

💡 Research Summary

The paper presents a novel low‑level algorithm for fast pattern matching in DNA sequences, addressing the limitations of a previously reported prime‑number‑based encoding method. The earlier approach encoded both the pattern and the text as products of distinct prime powers; matching was then reduced to a single multiplication and a comparison of the resulting integer. While conceptually elegant and fast for short patterns, this technique suffers from overflow and loss of precision when the pattern length grows beyond a few dozen bases, because the intermediate product quickly exceeds the capacity of standard 64‑bit or even 128‑bit integer types. Consequently, its practical applicability is restricted to relatively small patterns.

To overcome this bottleneck, the authors introduce two complementary innovations. First, they replace the unbounded integer multiplication with modular arithmetic. By selecting one or more large prime moduli and computing the product modulo each prime, the intermediate values remain bounded within the word size of the processor. The use of multiple, pairwise‑co‑prime moduli (a technique reminiscent of the Chinese Remainder Theorem) dramatically reduces the probability of false positives, because a collision would have to occur simultaneously in all modular domains. The modular products are then combined with a lightweight hash‑based verification step to confirm true matches. This approach eliminates overflow while preserving the one‑pass nature of the original algorithm.

Second, the algorithm is implemented at the assembly level to exploit modern SIMD extensions (AVX‑512, AVX2, SSE4.2, NEON) and to minimize instruction‑level overhead. The DNA alphabet (A, C, G, T) is encoded into two‑bit symbols and packed into 64‑bit words. A loop is unrolled to process multiple words per iteration, and modular multiplication is performed using Montgomery reduction, which is well suited to SIMD pipelines. By keeping all intermediate data in registers and avoiding memory accesses inside the hot loop, the authors achieve near‑theoretical throughput for the underlying hardware.

Parallelism is another cornerstone of the design. The reference genome is divided into fixed‑size chunks that are processed independently by worker threads. Each thread maintains its own local buffers, thereby avoiding false sharing and cache‑line contention. The implementation is NUMA‑aware: memory for each chunk is allocated on the node that runs the corresponding thread, reducing remote‑memory latency. A lightweight work‑stealing scheduler overlaps I/O (reading the next chunk from disk) with computation, ensuring that the CPU cores remain fully utilized even when the input data is streamed from storage.

The authors evaluate the method on the human genome (approximately 3 billion base pairs) using pattern sets ranging from 32 bp to 4096 bp. They compare against the original prime‑encoding algorithm and against a state‑of‑the‑art SIMD‑based string matcher (e.g., the “BNDM‑SIMD” implementation). Results show an average speed‑up of 18× over the prime‑based method and a 5–7× improvement over the SIMD matcher for patterns longer than 256 bp. Memory consumption is reduced to roughly 40 % of the baseline, because the modular representation requires only a few 64‑bit words per window rather than large arbitrary‑precision integers. Notably, for patterns of 1024 bp and longer, the original method fails due to overflow, whereas the proposed algorithm continues to operate correctly and efficiently.

The paper also discusses limitations. The choice of moduli influences the residual collision probability; while the authors select large 61‑bit primes to make this probability negligible, a formal analysis of worst‑case scenarios is not provided. Extremely long patterns (greater than 10 kb) still encounter bottlenecks related to register pressure and memory bandwidth, suggesting that further gains would require hierarchical tiling or off‑loading to GPUs or FPGAs.

Future work outlined by the authors includes dynamic modulus selection based on pattern length, integration with GPU‑accelerated modular multiplication, and scaling the approach to distributed clusters for whole‑genome indexing tasks.

In summary, the paper successfully extends the prime‑encoding concept to arbitrary pattern lengths by employing modular arithmetic and aggressive low‑level optimization. The combination of overflow‑safe arithmetic, SIMD‑driven computation, and careful multithreaded design yields substantial reductions in both runtime and memory usage, making the technique attractive for high‑throughput genomic analyses such as variant detection, motif scanning, and real‑time pathogen surveillance.