An Elementary Approach to Scheduling in Generative Diffusion Models

An elementary approach to characterizing the impact of noise scheduling and time discretization in generative diffusion models is developed. We first utilize the Cramér-Rao bound to identify the Gaussian setting as a fundamental performance limit, necessitating its study as a reference. Building on this insight, we consider a simplified model in which the source distribution is a multivariate Gaussian with a given covariance matrix, together with the deterministic reverse sampling process. The explicit closed-form evolution trajectory of the distributions across reverse sampling steps is derived, and consequently, the Kullback-Leibler (KL) divergence between the source distribution and the reverse sampling output is obtained. The effect of the number of time discretization steps on the convergence of this KL divergence is studied via the Euler-Maclaurin expansion. An optimization problem is formulated, and its solution noise schedule is obtained via calculus of variations, shown to follow a tangent law whose coefficient is determined by the eigenvalues of the source covariance matrix. For an alternative scenario, more realistic in practice, where pretrained models have been obtained for some given noise schedules, the KL divergence also provides a measure to compare different time discretization strategies in reverse sampling. Experiments across different datasets and pretrained models demonstrate that the time discretization strategy selected by our approach consistently outperforms baseline and search-based strategies, particularly when the budget on the number of function evaluations is very tight.

💡 Research Summary

This paper presents a principled, elementary analysis of how noise scheduling and time discretization affect the quality of samples generated by diffusion models (DMs). The authors begin by establishing that, under optimal score estimation, the Kullback‑Leibler (KL) divergence between the exact continuous‑time reverse stochastic differential equation (SDE) and its discretized counterpart attains a theoretical lower bound when the data distribution is Gaussian. This result is derived using the Girsanov theorem and the multivariate Cramér‑Rao bound, showing that the Gaussian case serves as a proxy for general distributions: any non‑Gaussian data will have a KL divergence at least as large as the Gaussian reference with the same covariance.

With this justification, the paper focuses on a simplified setting: the source data (x_0) follows a multivariate Gaussian (N(0,\Sigma_x)) and the reverse sampling is performed deterministically via the probability‑flow ODE (PF‑ODE). Because the forward diffusion is linear ((x_t = \alpha_t x_0 + \sigma_t \epsilon)), the optimal posterior mean is available in closed form, leading to an explicit expression for the deterministic reverse update. The authors prove that after (N) reverse steps the generated sample (\hat x_{t_0}) is also Gaussian with covariance (\mathbf{U},\mathrm{diag}(m_1,\dots,m_k),\mathbf{U}^\top), where (\mathbf{U}) diagonalizes (\Sigma_x) and each eigen‑dependent factor (m_\ell) is a product of ratios involving the noise schedule (\alpha_t,\sigma_t).

The KL divergence between the generated distribution (\hat p_{0,G}) and the true source (q_{0,G}) reduces to a sum over eigen‑directions: \