Injecting External Solutions Into CMA-ES

This report considers how to inject external candidate solutions into the CMA-ES algorithm. The injected solutions might stem from a gradient or a Newton step, a surrogate model optimizer or any other oracle or search mechanism. They can also be the result of a repair mechanism, for example to render infeasible solutions feasible. Only small modifications to the CMA-ES are necessary to turn injection into a reliable and effective method: too long steps need to be tightly renormalized. The main objective of this report is to reveal this simple mechanism. Depending on the source of the injected solutions, interesting variants of CMA-ES arise. When the best-ever solution is always (re-)injected, an elitist variant of CMA-ES with weighted multi-recombination arises. When \emph{all} solutions are injected from an \emph{external} source, the resulting algorithm might be viewed as \emph{adaptive encoding} with step-size control. In first experiments, injected solutions of very good quality lead to a convergence speed twice as fast as on the (simple) sphere function without injection. This means that we observe an impressive speed-up on otherwise difficult to solve functions. Single bad injected solutions on the other hand do no significant harm.

💡 Research Summary

The paper “Injecting External Solutions Into CMA‑ES” investigates how to incorporate candidate solutions that originate outside the standard sampling process of the Covariance Matrix Adaptation Evolution Strategy (CMA‑ES). CMA‑ES is a stochastic, black‑box optimizer that draws λ offspring from a multivariate normal distribution with mean mₜ, step‑size σₜ, and covariance matrix Cₜ. The algorithm then updates these parameters based on the ranking of the offspring. Because the update rules assume that the offspring are drawn from the current distribution, inserting arbitrary external points can break the adaptation mechanism, leading to premature stagnation or divergence.

The authors propose a minimal set of modifications that make CMA‑ES robust to such injections. The central idea is to renormalize any injected point so that its Mahalanobis distance to the current mean does not exceed a predefined bound. Concretely, for an injected offspring xᵢ the transformed deviation yᵢ = Cₜ⁻¹ᐟ² (xᵢ – mₜ) is clipped by a factor α_clip(c_y,‖yᵢ‖) = min(1, c_y/‖yᵢ‖) where c_y ≈ √n + 2 (n is the problem dimension). This operation (Equation 3) ensures that the length of the step is comparable to that of a typical internally sampled offspring, preserving the statistical assumptions underlying the covariance update.

A second, analogous clipping is applied to the mean shift Δm when the best‑ever solution is injected (Equation 6). By limiting the magnitude of Δm, the mean does not jump arbitrarily far, which would otherwise invalidate the evolution paths p_σ and p_c. The step‑size adaptation (Equation 11) is also constrained by a maximal allowed change Δ_max^σ (often set to 1.0), preventing σ from collapsing or exploding in response to a single large injected step.

Two injection regimes are discussed:

1. Partial injection – only a subset of the λ offspring are replaced by external solutions. The standard CMA‑ES parameters (population size, learning rates, etc.) can be kept unchanged; the clipping alone guarantees stability. This regime is suitable when an auxiliary optimizer (e.g., a local gradient descent) occasionally provides promising candidates.

2. Full injection – all offspring are supplied externally. In this case the algorithm resembles adaptive encoding: the external points are expressed directly in the phenotype space while the step‑size control remains active. The authors note that adaptive encoding typically discards step‑size adaptation, but their approach retains it, yielding a more flexible scheme.

A particularly interesting variant emerges when the current best‑ever solution is re‑injected at every generation. This yields an elitist CMA‑ES with weighted multi‑recombination, effectively combining the exploration power of CMA‑ES with the exploitation focus of an elite strategy. The authors argue that this “elitist with comma selection” can be implemented with negligible overhead.

Empirical validation is performed on the sphere function and the Rosenbrock function. Injecting a single high‑quality external point (a slightly perturbed optimum) accelerates convergence on the sphere function by roughly a factor of two, limited by the maximal step‑size reduction. On Rosenbrock, the speed‑up is even larger (up to a factor of nine), because the injected point provides a useful direction in a highly curved landscape. Conversely, injecting a poor-quality point has virtually no detrimental effect; the algorithm’s performance remains comparable to the baseline, confirming the robustness of the clipping mechanism.

Additional considerations include:

• Freezing certain coordinates (e.g., to enforce constraints) requires adjusting the effective dimension in the evolution path and step‑size formulas.
• The clipping function α_clip can be generalized by matching the empirical distribution of step lengths to a desired cumulative distribution function, allowing more sophisticated length control.
• Re‑injecting the same external solution multiple times is permissible and can be used to bias the search toward promising regions.

In summary, the paper demonstrates that only two simple safeguards—Mahalanobis‑distance clipping of injected steps and bounded step‑size changes—are sufficient to make CMA‑ES tolerant to arbitrary external proposals. This opens the door to hybrid optimization schemes where gradient information, surrogate‑model optima, repair mechanisms, or parallel meta‑heuristics can be seamlessly integrated into CMA‑ES without sacrificing its characteristic robustness and self‑adaptation. The approach is lightweight, requires no redesign of the core CMA‑ES loop, and shows promising empirical gains on benchmark problems, suggesting broad applicability in real‑world black‑box optimization scenarios where auxiliary information is available.