Analyzing and Guiding Zero-Shot Posterior Sampling in Diffusion Models

Recovering a signal from its degraded measurements is a long standing challenge in science and engineering. Recently, zero-shot diffusion based methods have been proposed for such inverse problems, offering a posterior sampling based solution that leverages prior knowledge. Such algorithms incorporate the observations through inference, often leaning on manual tuning and heuristics. In this work we propose a rigorous analysis of such approximate posterior-samplers, relying on a Gaussianity assumption of the prior. Under this regime, we show that both the ideal posterior sampler and diffusion-based reconstruction algorithms can be expressed in closed-form, enabling their thorough analysis and comparisons in the spectral domain. Building on these representations, we also introduce a principled framework for parameter design, replacing heuristic selection strategies used to date. The proposed approach is method-agnostic and yields tailored parameter choices for each algorithm, jointly accounting for the characteristics of the prior, the degraded signal, and the diffusion dynamics. We show that our spectral recommendations differ structurally from standard heuristics and vary with the diffusion step size, resulting in a consistent balance between perceptual quality and signal fidelity.

💡 Research Summary

This paper provides a rigorous theoretical framework for zero‑shot diffusion‑based inverse problem solvers, focusing on how to balance the diffusion prior with the measurement likelihood. The authors assume that the clean signal follows a multivariate Gaussian distribution (x_0\sim\mathcal N(\mu_0,\Sigma_0)). Under this assumption they derive a closed‑form optimal denoiser (Equation 9) that depends only on the prior covariance (\Sigma_0) and the diffusion noise schedule (\bar\alpha_t). This optimal denoiser coincides with the Bayes‑optimal estimator and serves as the foundation for all subsequent analysis.

The paper then examines the reverse diffusion process, specifically the deterministic DDIM sampler, and incorporates the measurement model (y=Hx_0+n) into the update rule. By projecting the update onto the Fourier basis and assuming that both the prior covariance and the degradation operator are shift‑invariant (hence diagonalizable in the Fourier domain), the authors obtain a set of independent scalar recursions for each frequency component. In this spectral domain the update can be written as a linear combination of three terms: (i) the noisy latent at the final diffusion step, (ii) the observed degraded measurement, and (iii) the prior mean. The coefficients of this combination—denoted (D_1, D_2, D_3)—are explicit functions of the noise schedule, the number of diffusion steps, and the eigenvalues (\lambda_i) of the prior covariance. Consequently, the entire diffusion trajectory can be interpreted as a frequency‑wise linear filter that merges prior information and data fidelity.

Having expressed the reconstruction in closed form, the authors turn to the central design problem: how to choose the guidance weights (\zeta_s) (or equivalently the variance‑based weight (r_s) used in other works) so that the distribution of the reconstructed signal matches the true posterior as closely as possible. They formulate an optimization problem that minimizes a discrepancy measure (D) between the estimated posterior (p(\hat x_F|y_F;\zeta,\bar\alpha)) and the exact posterior (p(x_F|y_F)). The discrepancy is taken to be the average Wasserstein‑2 distance over either a single observation or a set of observations. For the multi‑observation case the optimal weights admit a closed‑form solution (Equation 17) that depends only on the prior eigenvalues and the noise variance (\sigma_n^2). This result eliminates the need for heuristic tuning and yields guidance parameters that are intrinsically adapted to the prior, the degradation operator, and the diffusion schedule.

Experimental validation is performed on high‑resolution face images (FFHQ) and the ImageNet dataset. The authors compare their spectral‑based weight selection against common heuristics such as (\zeta_s = \zeta’ / |y-H\hat x_0|) and (r_s = \sqrt{1-\bar\alpha_s}). Across metrics including PSNR, SSIM, and LPIPS, the proposed method consistently outperforms the baselines, delivering higher fidelity to the measurements while preserving perceptual quality. Notably, the method remains robust when the number of diffusion steps is reduced, suggesting practical benefits for faster inference.

The paper’s contributions can be summarized as follows:

1. It unifies the ideal Bayesian posterior sampler and practical diffusion‑based zero‑shot samplers within a single spectral framework under a Gaussian prior.
2. It replaces heuristic guidance‑weight selection with a principled optimization based on Wasserstein‑2 distance, providing closed‑form solutions that are method‑agnostic.
3. It demonstrates that diffusion dynamics act as a frequency‑wise linear filter whose coefficients can be analytically expressed, enabling transparent analysis of the trade‑off between prior strength and data consistency.
4. It validates the theoretical findings with extensive experiments, showing improved reconstruction quality and reduced sensitivity to the number of diffusion steps.

Future work suggested by the authors includes extending the analysis to non‑Gaussian priors (e.g., mixture models or flow‑based priors), handling non‑linear measurement operators, and exploring alternative spectral bases such as wavelets. Moreover, integrating online adaptation of the guidance weights could further enhance robustness in real‑time or resource‑constrained settings. Overall, the paper advances the understanding of zero‑shot diffusion‑based inverse problem solving and provides a solid foundation for designing more principled and efficient algorithms.