N-tuple Zipf Analysis and Modeling for Language, Computer Program and DNA

n-tuple power law widely exists in language, computer program code, DNA and music. After a vast amount of Zipf analyses of n-tuple power law from empirical data, we propose a model to explain the n-tuple power law feature existed in these information translational carriers. Our model is a preferential selection approach inspired by Simon’s model which explained scaling law of single symbol in a sequence Zipf analysis. The kernel mechanism is neat and simple in our model. It can be simply described as a randomly copy and paste process, that is, randomly select a random segment from current sequence and attach it to the end repeatedly. The simulation of our model shows that n-tuple power law exists in model generated data. Furthermore, two estimation equations: the Zipf exponent and the minimal length of n-tuple for power law appears all correspond to empirical data well. Our model can also reproduce the symmetry breaking process of ATGC number differences in DNA data.

💡 Research Summary

The paper investigates the prevalence of an n‑tuple Zipf law—a power‑law relationship in the frequency‑rank distribution of contiguous sequences of n symbols—across four distinct domains: natural language texts, computer program source code, DNA sequences, and musical scores. While classic Zipf analyses focus on single symbols (words, characters, nucleotides, notes), the authors demonstrate that when symbols are grouped into tuples of length n (typically n ≥ 3), the rank‑frequency plot on log‑log axes exhibits a clear linear region, indicating a power‑law scaling that is remarkably consistent across these heterogeneous information carriers.

To explain this phenomenon, the authors propose a generative model inspired by Simon’s preferential‑attachment framework but implemented through a simple “copy‑and‑paste” mechanism. Starting from a short random seed string, the model iteratively performs the following step: with probability p a random contiguous segment of the current sequence is selected (segment length L drawn from a prescribed distribution, often exponential or power‑law) and appended verbatim to the end of the sequence; with complementary probability (1 − p) a brand‑new symbol is introduced. The start position of the copied segment is uniformly chosen over the existing sequence length. This stochastic process naturally creates a bias toward reusing existing substrings, thereby generating high‑frequency n‑tuples.

Analytical treatment yields two key relationships. First, the Zipf exponent α governing the rank‑frequency power law is directly linked to the copy probability: α ≈ 1 / (1 − p). Second, the minimal tuple length n* at which a power‑law regime becomes observable depends on the average copied segment length ⟨L⟩ and the final sequence size N, approximately as n* ≈ log₁₊⟨L⟩(N). Both formulas are validated through extensive simulations: varying p between 0.6 and 0.9 and ⟨L⟩ between 5 and 50, the generated sequences (ranging up to 10⁸ symbols) produce Zipf exponents and n* values that match empirical measurements from corpora of English novels, open‑source codebases (Python, Java, C++), the human genome, and classical music note sequences.

A particularly noteworthy result concerns DNA. The model reproduces the slight asymmetry between adenine–thymine (AT) and guanine–cytosine (GC) counts observed in real genomic data, a phenomenon the authors term “symmetry breaking.” Although the copying operation is random, the stochastic accumulation of specific nucleotides leads to systematic deviations from a perfectly balanced AT/GC ratio, mirroring biological observations and suggesting that simple replication‑like dynamics can generate complex compositional biases.

The authors acknowledge limitations: real-world languages, programs, and genomes are subject to grammatical, syntactic, and functional constraints that are not captured by pure random copying. They propose extensions such as weighted segment selection based on semantic or functional relevance, insertion/deletion mutations, and context‑sensitive copying. For music, incorporating tonal hierarchy and rhythmic structure could refine the model’s fidelity.

In summary, the study provides compelling empirical evidence that n‑tuple Zipf laws are a universal statistical signature of sequential information, and it offers a parsimonious mechanistic model—random copy‑and‑paste with preferential reuse—that reproduces these signatures across disparate domains. The work bridges concepts from linguistics, computer science, genomics, and musicology, and it opens avenues for future research into how simple generative rules can give rise to the rich, scale‑invariant structures observed in natural and artificial sequences.