Theory of Speciation Transitions in Diffusion Models with General Class Structure

Diffusion Models generate data by reversing a stochastic diffusion process, progressively transforming noise into structured samples drawn from a target distribution. Recent theoretical work has shown that this backward dynamics can undergo sharp qualitative transitions, known as speciation transitions, during which trajectories become dynamically committed to data classes. Existing theoretical analyses, however, are limited to settings where classes are identifiable through first moments, such as mixtures of Gaussians with well-separated means. In this work, we develop a general theory of speciation in diffusion models that applies to arbitrary target distributions admitting well-defined classes. We formalize the notion of class structure through Bayes classification and characterize speciation times in terms of free-entropy difference between classes. This criterion recovers known results in previously studied Gaussian-mixture models, while extending to situations in which classes are not distinguishable by first moments and may instead differ through higher-order or collective features. Our framework also accommodates multiple classes and predicts the existence of successive speciation times associated with increasingly fine-grained class commitment. We illustrate the theory on two analytically tractable examples: mixtures of one-dimensional Ising models at different temperatures and mixtures of zero-mean Gaussians with distinct covariance structures. In the Ising case, we obtain explicit expressions for speciation times by mapping the problem onto a random-field Ising model and solving it via the replica method. Our results provide a unified and broadly applicable description of speciation transitions in diffusion-based generative models.

💡 Research Summary

Diffusion models (DMs) have become the dominant framework for high‑fidelity generative tasks, yet the dynamics of their reverse‑time stochastic process remain only partially understood. Recent work uncovered “speciation transitions,” sharp symmetry‑breaking events during which a trajectory becomes committed to a particular data class. Existing analyses, however, are confined to mixtures of Gaussians whose components are separated by their first moments (means). The present paper lifts this restriction and builds a unified theory of speciation that applies to any target distribution possessing a well‑defined class structure, even when classes differ only in higher‑order statistics or collective features.

The authors begin by formalizing a Proper Density Decomposition of the target distribution (P(a)=\sum_{r=1}^{R} w_r P_r(a)). Each component (P_r) is required to be a pure density: in the high‑dimensional limit, a Bayes classifier can assign a noisy observation (\tilde a_r = a_r + \eta) (with (\eta = O_N(1))) to the correct component with probability (1-\varepsilon(N)), where (\varepsilon(N)=o_N(1)). This captures the intuitive notion that a typical sample belongs unambiguously to a class, even if the separation is not visible in the mean.

To study speciation, the paper introduces a time‑dependent Bayes classifier for forward‑diffused points (x). At diffusion time (t) the conditional probability is
\