Diffusion Models in Simulation-Based Inference: A Tutorial Review

Diffusion models have recently emerged as powerful learners for simulation-based inference (SBI), enabling fast and accurate estimation of latent parameters from simulated and real data. Their score-based formulation offers a flexible way to learn conditional or joint distributions over parameters and observations, thereby providing a versatile solution to various modeling problems. In this tutorial review, we synthesize recent developments on diffusion models for SBI, covering design choices for training, inference, and evaluation. We highlight opportunities created by various concepts such as guidance, score composition, flow matching, consistency models, and joint modeling. Furthermore, we discuss how efficiency and statistical accuracy are affected by noise schedules, parameterizations, and samplers. Finally, we illustrate these concepts with case studies across parameter dimensionalities, simulation budgets, and model types, and outline open questions for future research.

💡 Research Summary

This tutorial review provides a comprehensive synthesis of recent advances that integrate diffusion models into simulation‑based inference (SBI). The authors begin by framing SBI as the intersection of three essential components: a simulator that can generate synthetic observations y from latent parameters θ, a prior distribution p(θ), and an approximator that learns to invert the simulator. Traditional SBI approaches have relied on kernel density estimation, normalizing flows, or GANs, but diffusion models bring a fundamentally different, score‑based formulation. By learning to progressively denoise a noisy latent variable, diffusion models can represent conditional or joint distributions p(θ, y) without requiring an explicit likelihood, making them naturally suited to the “doubly‑intractable” problems that arise when only a stochastic simulator is available.

The review categorises inference targets into four families—prior, likelihood, posterior, and joint distribution—and discusses the trade‑offs of learning each with diffusion models. Direct posterior learning yields amortised inference but requires retraining if the prior changes; likelihood learning decouples prior specification but necessitates an additional MCMC step for posterior sampling; joint learning can simultaneously address simulation‑to‑real gaps and support multi‑task or multi‑modal scenarios.

A major contribution of the paper is a systematic analysis of design choices that affect both statistical accuracy and computational efficiency. First, the choice of noise schedule (linear, cosine, or reverse‑time) influences the number of sampling steps required for a given level of approximation error. Second, parameterisation strategies—ε‑score versus x‑score, and hybrid schemes that combine diffusion with variational auto‑encoders—are examined for their ability to map high‑dimensional observations (e.g., images, graphs) to low‑dimensional parameters. Third, the selection of samplers (DDIM, PNDM, high‑order Langevin) is compared, showing that fewer deterministic steps can preserve posterior fidelity when the simulation budget is tight.

The authors also highlight several SBI‑specific augmentations to the vanilla diffusion framework. Conditional guidance injects external gradient information (e.g., from a physics‑based model) into the sampling trajectory, improving sample quality under limited simulation budgets. Score composition combines multiple score networks, enabling the model to satisfy several objectives such as multi‑modality or multi‑scale consistency simultaneously. Flow‑matching and consistency models reduce training‑time back‑propagation costs, which is crucial for domains where each simulation run may take hours or days (e.g., fluid dynamics, cosmological N‑body simulations).

Empirical results span a broad spectrum of problem settings: low‑dimensional toy models (Gaussian mixtures, two‑moons), high‑dimensional image‑based inference (astronomical imaging, ultrasound), Bayesian neural networks, and large‑scale N‑body simulations. Across these benchmarks, diffusion‑based SBI consistently outperforms prior methods in KL divergence, coverage probability, and calibration metrics, while often requiring fewer simulation calls. Notably, in high‑cost simulation regimes, a hybrid strategy—training a diffusion‑based likelihood model for low‑dimensional θ and a conditional diffusion model for high‑dimensional y—achieves the best trade‑off between accuracy and computational load.

The review concludes by outlining open challenges: multi‑fidelity training that leverages simulations of varying accuracy, online or incremental diffusion training for streaming simulation data, and domain adaptation techniques to bridge the gap between simulated and real observations. Addressing these issues will further cement diffusion models as a versatile, high‑performance backbone for SBI across scientific and engineering disciplines.