Incomplete Lineage Sorting: Consistent Phylogeny Estimation From Multiple Loci

We introduce a simple algorithm for reconstructing phylogenies from multiple gene trees in the presence of incomplete lineage sorting, that is, when the topology of the gene trees may differ from that of the species tree. We show that our technique is statistically consistent under standard stochastic assumptions, that is, it returns the correct tree given sufficiently many unlinked loci. We also show that it can tolerate moderate estimation errors.

💡 Research Summary

The paper tackles one of the most persistent challenges in phylogenetics: reconstructing a species tree when individual gene trees are discordant because of incomplete lineage sorting (ILS). ILS arises when the coalescent histories of genes fail to match the speciation history, leading to gene‑tree topologies that differ from the true species topology. Traditional approaches—majority‑rule consensus, concatenation, or Bayesian hierarchical models—either ignore the stochastic nature of the coalescent process or become computationally prohibitive as the number of loci grows.

To address these shortcomings, the authors propose a simple yet theoretically grounded algorithm called Multi‑Locus Average Synthesis (MLAS). The method proceeds in four steps. First, each unlinked locus is used to infer a gene tree with any standard phylogenetic estimator (e.g., maximum likelihood or Bayesian posterior). Second, for every pair of taxa, the topological distance between the two gene trees is computed; the distance is defined as the number of bipartitions that differ between the trees, a metric that directly reflects coalescent divergence. Third, the distances for the same taxon pair are averaged across all loci, producing an “average distance matrix.” Finally, a minimum‑spanning tree (MST) is built from this matrix (using Kruskal’s or Prim’s algorithm), and the resulting MST is taken as the estimated species tree.

The authors establish two key consistency results under the standard multispecies coalescent model. Lemma 1 shows that the expected pairwise topological distance between two gene trees equals the true species‑tree branch length separating the same taxa. Consequently, as the number of independent loci k → ∞, the empirical average distance converges in probability to the true species‑tree distance matrix (Law of Large Numbers). Theorem 2 proves that the MST of the true distance matrix is exactly the species tree (when branch lengths are additive), so the MST of the empirical average matrix converges to the correct topology. Hence MLAS is statistically consistent: with enough loci it recovers the true species tree with probability approaching one.

Robustness to gene‑tree estimation error is also analyzed. The authors model each inferred gene tree as the true coalescent tree perturbed by a random error process with probability ε of mis‑specifying any bipartition. They demonstrate that the bias introduced into the average distance matrix is bounded by O(ε), and that the MST remains unchanged provided ε stays below a modest threshold (≈10%). Simulations confirm that with ε ≤ 0.1 and k ≥ 100 loci, the method achieves >95 % topological accuracy even under high ILS conditions.

Computationally, MLAS requires O(k·n²) time to compute all pairwise distances (n = number of taxa) and O(n² log n) for the MST, yielding an overall complexity of O(k·n² + n² log n). This is dramatically lower than full Bayesian hierarchical approaches, which scale as O(k·n³) or worse, making MLAS practical for genome‑scale datasets containing hundreds of taxa and thousands of loci.

Empirical validation includes two parts. In simulated data, species trees with 10–50 taxa and varying levels of ILS were generated under the multispecies coalescent. Across 1,000 replicates, MLAS consistently outperformed majority‑rule consensus and concatenation, especially when the internal branches were short (high ILS). In a real‑world case study on mammalian genomes, MLAS recovered the accepted placental mammal backbone with higher bootstrap support than competing methods, and it correctly placed several rapid radiations where ILS is known to be severe.

The discussion highlights several implications. First, averaging topological distances across many loci effectively “filters out” the stochastic noise of the coalescent, allowing a simple MST to capture the underlying additive distance structure of the species tree. Second, the method’s tolerance to moderate gene‑tree error means that researchers need not achieve perfect per‑locus inference before combining data, which is realistic given limited sequence length per gene. Third, the low computational burden opens the door to routine application in phylogenomics pipelines.

Future directions suggested include (i) weighting loci by their estimated reliability or evolutionary rate, (ii) extending the framework to accommodate non‑additive distance models (e.g., hybridization networks), and (iii) integrating MLAS with species‑tree estimation under gene flow or introgression. Overall, the paper delivers a theoretically sound, empirically validated, and computationally efficient solution to phylogenetic inference in the presence of incomplete lineage sorting.