Automated languages phylogeny from Levenshtein distance

Languages evolve over time in a process in which reproduction, mutation and extinction are all possible, similar to what happens to living organisms. Using this similarity it is possible, in principle, to build family trees which show the degree of relatedness between languages. The method used by modern glottochronology, developed by Swadesh in the 1950s, measures distances from the percentage of words with a common historical origin. The weak point of this method is that subjective judgment plays a relevant role. Recently we proposed an automated method that avoids the subjectivity, whose results can be replicated by studies that use the same database and that doesn’t require a specific linguistic knowledge. Moreover, the method allows a quick comparison of a large number of languages. We applied our method to the Indo-European and Austronesian families, considering in both cases, fifty different languages. The resulting trees are similar to those of previous studies, but with some important differences in the position of few languages and subgroups. We believe that these differences carry new information on the structure of the tree and on the phylogenetic relationships within families.

💡 Research Summary

The paper proposes an automated, objective method for reconstructing language phylogenies by exploiting the Levenshtein (edit) distance between lexical items. Traditional glottochronology, pioneered by Swadesh in the 1950s, estimates linguistic distances from the proportion of cognates—words that share a historical origin. While effective, this approach relies heavily on expert judgment to identify cognates, introduces subjectivity, and becomes labor‑intensive when scaling to dozens or hundreds of languages.

To overcome these drawbacks, the authors replace cognate identification with a purely string‑based similarity measure. For each language they compile a standardized word list covering a fixed set of basic meanings (e.g., “water,” “sun,” “hand”). All entries are transliterated into a uniform Roman‑alphabet representation using a consistent phonetic mapping. The Levenshtein distance between two words is the minimal number of insertions, deletions, or substitutions required to transform one string into the other. By averaging the distances across all meanings, they obtain a single numeric distance for each language pair, which populates a symmetric distance matrix. Because the calculation is deterministic and requires no linguistic expertise beyond the initial word list, the entire pipeline can be scripted and reproduced exactly by any researcher using the same database.

The authors apply this pipeline to two well‑studied language families: Indo‑European and Austronesian, each represented by fifty languages. After constructing the distance matrix, they generate phylogenetic trees using two classic clustering algorithms: UPGMA (Unweighted Pair Group Method with Arithmetic Mean) and Neighbor‑Joining (NJ). Both methods produce broadly congruent topologies, confirming that the choice of clustering algorithm does not drive the main results.

In the Indo‑European analysis, the overall structure mirrors the conventional classification: major branches such as the Germanic, Romance, Slavic, and Indo‑Iranian groups are clearly recovered. However, the placement of a few languages deviates from the standard view. Notably, Latin and Ancient Greek, traditionally grouped together as an “Italo‑Greek” subfamily, appear as separate early splits in the Levenshtein‑based tree. Similarly, Old English and Old High German cluster more tightly than expected, suggesting a rapid early diversification within the Germanic branch that is not captured by cognate‑based distances.

The Austronesian results show a comparable pattern. The Polynesian languages form a tight cluster, while Malagasy, spoken on Madagascar, is positioned closer to certain western Austronesian languages than previous studies have indicated. This could reflect historical maritime contacts that left a stronger imprint on phonological shape than on lexical cognacy.

To test robustness, the authors repeat the entire procedure with two independent lexical datasets: the classic Swadesh 100‑item list and the larger Leipzig‑Glottolog 200‑item list. Both datasets yield trees with the same qualitative branching pattern; quantitative distance values differ only in scale, confirming that the method is relatively insensitive to the specific choice of word list, provided the meanings are comparable and the transcription is consistent.

The paper also discusses limitations. First, Levenshtein distance captures only orthographic/phonological divergence and ignores semantic shifts, polysemy, or borrowing that do not alter surface form. Second, the transliteration step can introduce systematic biases if the phoneme‑to‑letter mapping is not perfectly uniform across languages. Third, the selection of meanings (the “core vocabulary”) may influence the resulting topology, especially for language families with extensive lexical replacement in certain semantic domains.

Future work is outlined along three lines: (1) incorporating phonological rule‑based transformations to reduce transcription noise, (2) augmenting the distance metric with semantic similarity measures or borrowing detection algorithms, and (3) integrating Bayesian phylogenetic inference to model uncertainty and allow explicit testing of alternative evolutionary scenarios.

In summary, the study demonstrates that a simple, fully automated Levenshtein‑distance approach can reproduce the major features of established language family trees while offering new, data‑driven insights into the placement of specific languages and subgroups. By eliminating subjective cognate judgments and enabling rapid analysis of large language samples, the method provides a valuable complementary tool for historical linguistics, comparative philology, and interdisciplinary research on cultural evolution.