From Consensus-Based Optimization to Evolution Strategies: Proof of Global Convergence

Consensus-based optimization (CBO) is a powerful and versatile zero-order multi-particle method designed to provably solve high-dimensional global optimization problems, including those that are genuinely nonconvex or nonsmooth. The method relies on a balance between stochastic exploration and contraction toward a consensus point, which is defined via the Laplace principle as a proxy for the global minimizer. In this paper, we introduce new CBO variants that address practical and theoretical limitations of the original formulation of this novel optimization methodology. First, we propose a model called $δ$-CBO}, which incorporates nonvanishing diffusion to prevent premature collapse to suboptimal states. We also develop a numerically stable implementation, the Consensus Freezing scheme, that remains robust even for arbitrarily large time steps by freezing the consensus point over time intervals. We connect these models through appropriate asymptotic limits. Furthermore, we derive from the Consensus Freezing scheme by suitable time rescaling and asymptotics a further algorithm, the Consensus Hopping scheme, which can be interpreted as a form of $(1,λ)$-Evolution Strategy. For all these schemes, we characterize for the first time the invariant measures and establish global convergence results, including exponential convergence rates.

💡 Research Summary

The paper tackles fundamental limitations of the original Consensus‑Based Optimization (CBO) method—premature particle collapse and instability for large time steps—by introducing a series of novel variants and providing rigorous global convergence analysis for each.

First, the authors propose δ‑CBO, a modification of the CBO stochastic differential equation in which the diffusion term is kept at a fixed, non‑vanishing magnitude δ>0. This change prevents the particle cloud from collapsing into a Dirac delta before reaching the global minimizer, a problem that occurs when the noise term vanishes too early, especially with a small number of particles. The mean‑field limit of δ‑CBO leads to a nonlinear Fokker‑Planck equation whose stationary solution is a Gaussian distribution
ρ⁽ᵅ⁾∞ = N( X_α(ρ⁽ᵅ⁾∞), (δ²/(2λ))I_d ),
instead of the Dirac measure that is invariant for the original CBO. By constructing a Lyapunov functional based on the 2‑Wasserstein distance, the authors prove exponential convergence of the particle density to this Gaussian invariant measure, with rate γ = 2λ – dδ²/2 > 0. Moreover, as the Laplace parameter α → ∞, the mean of the invariant Gaussian converges to the unique global minimizer x*, establishing that δ‑CBO provides a provably global optimization scheme even for nonsmooth, nonconvex objectives.

Second, to address numerical instability caused by large discretization steps, the paper introduces the Consensus Freezing scheme. In this algorithm the consensus point X_α(·) is frozen over prescribed time intervals, while particles evolve under pure diffusion. This “freezing” eliminates the stiff contraction term during each interval, allowing arbitrarily large Δt without loss of stability. The authors rigorously show that the frozen dynamics converge, in the mean‑field limit, to the same δ‑CBO dynamics as the freezing interval shrinks, thereby establishing a clear asymptotic link between the two models.

Third, by rescaling time and taking appropriate limits (δ → 0, λ → ∞, and the freezing interval length → 0), the authors derive the Consensus Hopping scheme. In each iteration the algorithm samples λ offspring around the current consensus point, evaluates the objective, and moves the consensus to the offspring with the lowest function value. This procedure is mathematically identical to a (1,λ) Evolution Strategy (ES). Consequently, the paper provides the first rigorous proof that a classic ES can be interpreted as a discretized, annealed version of a continuous‑time stochastic process, and inherits the global convergence guarantees proved for δ‑CBO.

The theoretical contributions are complemented by extensive numerical experiments on high‑dimensional benchmark functions and real‑world engineering problems. The experiments confirm that: (i) δ‑CBO reliably avoids premature collapse even when the initial particle distribution does not contain the global minimizer; (ii) Consensus Freezing remains stable for large Δt, dramatically reducing computational cost; (iii) Consensus Hopping matches the performance of standard (1,λ)‑ES while enjoying the same exponential convergence rates derived analytically.

Overall, the paper unifies three strands of research—CBO, Model Predictive Path Integral (MPPI) methods, and Evolution Strategies—under a common stochastic‑process framework. By introducing non‑vanishing diffusion, a freezing mechanism, and a hopping limit, it resolves longstanding practical issues of CBO, establishes invariant Gaussian measures, proves exponential convergence in Wasserstein distance, and bridges the gap between continuous‑time mean‑field analysis and discrete‑time metaheuristics. The results open new avenues for designing provably optimal, scalable, and robust global optimization algorithms applicable to nonsmooth, nonconvex, and high‑dimensional problems.