Languages cool as they expand: Allometric scaling and the decreasing need for new words

We analyze the occurrence frequencies of over 15 million words recorded in millions of books published during the past two centuries in seven different languages. For all languages and chronological subsets of the data we confirm that two scaling regimes characterize the word frequency distributions, with only the more common words obeying the classic Zipf law. Using corpora of unprecedented size, we test the allometric scaling relation between the corpus size and the vocabulary size of growing languages to demonstrate a decreasing marginal need for new words, a feature that is likely related to the underlying correlations between words. We calculate the annual growth fluctuations of word use which has a decreasing trend as the corpus size increases, indicating a slowdown in linguistic evolution following language expansion. This “cooling pattern” forms the basis of a third statistical regularity, which unlike the Zipf and the Heaps law, is dynamical in nature.

💡 Research Summary

The paper investigates statistical regularities in the evolution of large‑scale written language by analyzing more than 15 million distinct word types drawn from millions of books published over the past two centuries in seven major languages (English, French, German, Spanish, Italian, Russian, and Chinese). Using the Google Books Ngram corpus, the authors first confirm that the classic Zipf law (frequency ∝ rank⁻¹) holds only for the most frequent words. When the full frequency spectrum is examined, two distinct scaling regimes emerge: a Zipfian regime for high‑frequency tokens and a steeper regime (exponent ≈ 1.5–2.0) for mid‑ and low‑frequency words, indicating that different generative mechanisms govern common versus rare vocabulary.

Next, the relationship between corpus size (N, total token count) and vocabulary size (V, number of unique word types) is explored. While Heaps’ law (V ∝ N^β) is reproduced with β values ranging from 0.5 to 0.7, the authors demonstrate an allometric scaling pattern in which β systematically declines as N grows. In corpora exceeding 10⁸ tokens, β falls below 0.55, revealing a diminishing marginal return for new word creation: as a language expands, existing words are reused in increasingly diverse contexts, reducing the need for novel lexical items.

The most novel contribution is the analysis of temporal fluctuations in word usage. For each year the authors compute the relative change Δf/f for every word and measure its standard deviation σ(N). They find σ scales as N^(–α) with α ≈ 0.1–0.2, meaning that larger corpora exhibit smaller year‑to‑year variability. This “cooling” effect implies that linguistic evolution slows down as the language’s written body expands, establishing a third, dynamical statistical regularity that complements the static Zipf and Heaps laws.

The discussion interprets the cooling pattern as a signature of language behaving like a complex network: high‑frequency words form a tightly connected core, while low‑frequency words occupy peripheral positions. As the network grows, its topology becomes more robust, limiting the impact of stochastic fluctuations and curbing the introduction of new nodes (words). The authors argue that this insight has practical implications for natural‑language‑processing systems and language‑policy planning. In large‑scale NLP models, indiscriminate vocabulary expansion may be inefficient; instead, strategies that respect the observed diminishing marginal utility and reduced volatility could improve model compactness and stability.

In summary, the study provides robust empirical evidence that language exhibits three intertwined scaling laws: (1) Zipfian scaling for common words, (2) an allometric Heaps‑type relation showing decreasing marginal need for new words, and (3) a dynamical cooling law where growth fluctuations shrink with corpus size. Together, these findings deepen our understanding of how written language self‑organizes and evolves, and they open new avenues for modeling linguistic dynamics in both theoretical and applied contexts.