Optimal Convergence Analysis of DDPM for General Distributions

Score-based diffusion models have achieved remarkable empirical success in generating high-quality samples from target data distributions. Among them, the Denoising Diffusion Probabilistic Model (DDPM) is one of the most widely used samplers, generating samples via estimated score functions. Despite its empirical success, a tight theoretical understanding of DDPM – especially its convergence properties – remains limited. In this paper, we provide a refined convergence analysis of the DDPM sampler and establish near-optimal convergence rates under general distributional assumptions. Specifically, we introduce a relaxed smoothness condition parameterized by a constant $L$, which is small for many practical distributions (e.g., Gaussian mixture models). We prove that the DDPM sampler with accurate score estimates achieves a convergence rate of $$\widetilde{O}\left(\frac{d\min{d,L^2}}{T^2}\right)~\text{in Kullback-Leibler divergence},$$ where $d$ is the data dimension, $T$ is the number of iterations, and $\widetilde{O}$ hides polylogarithmic factors in $T$. This result substantially improves upon the best-known $d^2/T^2$ rate when $L < \sqrt{d}$. By establishing a matching lower bound, we show that our convergence analysis is tight for a wide array of target distributions. Moreover, it reveals that DDPM and DDIM share the same dependence on $d$, raising an interesting question of why DDIM often appears empirically faster.

💡 Research Summary

This paper presents a refined and near-optimal convergence analysis for the Denoising Diffusion Probabilistic Model (DDPM), a cornerstone sampler in score-based diffusion models. It addresses a significant gap between DDPM’s empirical success and its limited theoretical understanding, particularly regarding convergence rates under realistic assumptions.

The core innovation lies in introducing a relaxed smoothness condition, termed the “non-uniform Lipschitz property” (Definition 1). Unlike the commonly used global Lipschitz assumption on the score functions—which can be excessively large or infinite for complex distributions—this new condition bounds the scaled gradient τ∇s*_τ(X_τ) with high probability by a parameter L. Crucially, L is shown to be small (e.g., polylogarithmic) for many practical distributions like Gaussian mixtures, while being much more permissive than global smoothness.

Under this relaxed condition and a minimal second-moment assumption on the target data distribution, the paper establishes sharp convergence guarantees for the DDPM sampler with accurate score estimates (Theorem 1). The key results are:

• In Total Variation (TV) distance, the convergence rate is (\widetilde{O}\left( \frac{\sqrt{d} \cdot \min{\sqrt{d}, L}}{T} \right)).
• In Kullback-Leibler (KL) divergence, the rate is (\widetilde{O}\left( \frac{d \cdot \min{d, L^2}}{T^2} \right)).

When (L < \sqrt{d})—a scenario common in practice—these rates substantially improve upon the previously best-known (\widetilde{O}(d/T)) TV rate and (\widetilde{O}(d^2/T^2)) KL rate. Notably, the analysis reveals that DDPM achieves the same (\sqrt{d}) dimensional dependence as the Denoising Diffusion Implicit Model (DDIM), challenging the perception of DDPM’s inherent inefficiency and raising questions about the empirical factors behind DDIM’s observed speed.

To demonstrate the tightness of these upper bounds, the paper provides a matching lower bound (Theorem 2). It proves that for a Gaussian target distribution and standard learning rate schedules, the KL divergence of the DDPM output is at least (\Omega(d/T^2)). This confirms that the derived convergence rates are essentially optimal for a broad class of problems.

The technical proof is built upon a novel auxiliary reverse process construction that shares the same marginal distributions as the forward noising process. By comparing the DDPM sampler to this auxiliary process, the analysis cleanly decomposes the error into discretization error (from approximating a continuous-time ODE) and score estimation error. The non-uniform Lipschitz property is then skillfully leveraged to control the discretization error tightly, leading to the final refined rates.

In summary, this work provides a fundamental theoretical advancement in understanding diffusion models. It offers rigorous, near-optimal convergence guarantees for DDPM under realistic and general distributional assumptions, bridges the theoretical gap between DDPM and DDIM, and provides a robust analytical framework for future research into the design and analysis of diffusion samplers.