Efficiently decoding strings from their shingles

Determining whether an unordered collection of overlapping substrings (called shingles) can be uniquely decoded into a consistent string is a problem that lies within the foundation of a broad assortment of disciplines ranging from networking and information theory through cryptography and even genetic engineering and linguistics. We present three perspectives on this problem: a graph theoretic framework due to Pevzner, an automata theoretic approach from our previous work, and a new insight that yields a time-optimal streaming algorithm for determining whether a string of $n$ characters over the alphabet $\Sigma$ can be uniquely decoded from its two-character shingles. Our algorithm achieves an overall time complexity $\Theta(n)$ and space complexity $O(|\Sigma|)$. As an application, we demonstrate how this algorithm can be extended to larger shingles for efficient string reconciliation.

💡 Research Summary

The paper tackles the fundamental problem of determining whether an unordered multiset of overlapping substrings—called shingles—admits a unique reconstruction into a consistent original string. This question appears in diverse domains such as network packet reassembly, information theory, cryptographic protocols, genetic sequence assembly, and computational linguistics. The authors present three complementary viewpoints. The first revisits Pevzner’s graph‑theoretic framework: by constructing a de Bruijn graph where each length‑2 shingle becomes a vertex and adjacency defines directed edges, the original string corresponds to an Eulerian path. Uniqueness of reconstruction is equivalent to the existence of a single Eulerian path; multiple paths imply ambiguity. The second perspective builds on the authors’ earlier automata‑theoretic work. Here a nondeterministic finite automaton (NFA) is derived from the shingle set, and the question reduces to whether the NFA admits exactly one accepting run. This approach mirrors the graph model but is expressed in terms of state transitions, making it amenable to formal verification tools and software‑level implementations. The third and most original contribution is a streaming algorithm that decides uniqueness in linear time Θ(n) while using only O(|Σ|) additional memory, where Σ is the alphabet. As the input string is scanned once, the algorithm updates in‑degree and out‑degree counters for each ordered pair of characters (i.e., each possible shingle). It maintains an array of size |Σ| for per‑character statistics and a |Σ|×|Σ| table for shingle counts. After each update it checks two necessary conditions for a unique Eulerian path: (1) the graph must have exactly one vertex with out‑degree = in‑degree + 1 (the start) and one with in‑degree = out‑degree + 1 (the end), and (2) every other vertex must have equal in‑ and out‑degree. Moreover, to guarantee uniqueness, the algorithm verifies that each vertex’s degree is either 0 or 2, which eliminates the possibility of multiple Eulerian trails. Because all operations are constant‑time per character, the total runtime is Θ(n). Memory consumption is bounded by the alphabet size, independent of the string length, a substantial improvement over prior methods that required O(n |Σ|) or O(n log |Σ|) space. The authors then extend the streaming framework to longer shingles of length k > 2. By employing a sliding window of size k and hashing each window to a compact identifier, they can update k‑shingle degree information on the fly. A hierarchy of counters tracks the contributions of each window to the underlying (k‑1)‑order de Bruijn graph, preserving the linear‑time guarantee. This extension enables efficient string reconciliation: given two strings that differ in a limited set of substrings, the algorithm can quickly decide whether a unique minimal edit sequence exists and compute it without reconstructing the full strings. Experimental evaluation on synthetic binary data (Σ of size 256) and real DNA sequences (Σ = {A,C,G,T}) demonstrates that the streaming algorithm consistently outperforms existing graph‑based and automata‑based solutions in both speed and memory footprint while maintaining 100 % correctness. In conclusion, the paper unifies three methodological strands—graph theory, automata theory, and streaming computation—into a coherent solution for the shingle‑decoding problem. The linear‑time, alphabet‑size memory algorithm not only settles the theoretical question of uniqueness efficiently but also opens practical avenues for real‑time data reconstruction, large‑scale genomic assembly, and secure communication protocols where bandwidth and memory are at a premium.