Adaptive hybrid optimization strategy for calibration and parameter estimation of physical models

A new adaptive hybrid optimization strategy, entitled squads, is proposed for complex inverse analysis of computationally intensive physical models. The new strategy is designed to be computationally efficient and robust in identification of the global optimum (e.g. maximum or minimum value of an objective function). It integrates a global Adaptive Particle Swarm Optimization (APSO) strategy with a local Levenberg-Marquardt (LM) optimization strategy using adaptive rules based on runtime performance. The global strategy optimizes the location of a set of solutions (particles) in the parameter space. The LM strategy is applied only to a subset of the particles at different stages of the optimization based on the adaptive rules. After the LM adjustment of the subset of particle positions, the updated particles are returned to the APSO strategy. The advantages of coupling APSO and LM in the manner implemented in squads is demonstrated by comparisons of squads performance against Levenberg-Marquardt (LM), Particle Swarm Optimization (PSO), Adaptive Particle Swarm Optimization (APSO; the TRIBES strategy), and an existing hybrid optimization strategy (hPSO). All the strategies are tested on 2D, 5D and 10D Rosenbrock and Griewank polynomial test functions and a synthetic hydrogeologic application to identify the source of a contaminant plume in an aquifer. Tests are performed using a series of runs with random initial guesses for the estimated (function/model) parameters. Squads is observed to have the best performance when both robustness and efficiency are taken into consideration than the other strategies for all test functions and the hydrogeologic application.

💡 Research Summary

The paper introduces “Squads,” an adaptive hybrid optimization framework that couples Adaptive Particle Swarm Optimization (APSO) with the Levenberg‑Marquardt (LM) local optimizer to address the challenges of inverse modeling for computationally intensive physical systems. Traditional global optimizers (e.g., PSO) excel at exploring complex, multimodal objective surfaces but often require many function evaluations to converge, while local methods such as LM converge rapidly once they are near a basin of attraction but lack robustness against multiple minima. Squads leverages the TRIBES variant of APSO, which organizes particles into hierarchical groups called “tribes.” Within each tribe, the particle with the best objective value is designated the “shaman,” and the best shaman across the entire swarm guides inter‑tribe information exchange.

The novelty lies in a set of adaptive rules that dynamically decide when to invoke LM on selected shamans and when to modify the tribe structure. After each iteration, the algorithm evaluates the improvement ratio of particles within a tribe; if a sufficient fraction of particles have improved, the tribe is deemed “good” and the worst particle is removed, focusing computational effort on promising regions. Conversely, “bad” tribes receive new particles or random dimension re‑initializations to boost exploration. The swarm as a whole is flagged as “bad” when recent LM updates fail to reduce the objective by at least two‑thirds, prompting another LM call on the current best shaman. LM updates are performed using the LevMar library, which estimates Jacobian and Hessian information automatically, and the adjusted particle is fed back into the APSO loop. This interplay ensures that LM provides rapid local refinement without monopolizing the search, while APSO maintains global diversity.

The algorithm’s flow is illustrated in Figure 1. Initialization can start with the standard 10 + 2√D mono‑particle tribes or with a single tribe that expands adaptively. Particle positions are generated via user‑specified values, uniform random sampling, or sampling within a hyper‑parallelepiped surrounding the current global best. Velocity updates follow the classic PSO equation with inertia, cognitive, and social components, but the parameters are self‑tuned by the TRIBES mechanism, eliminating manual tuning.

Performance is assessed on benchmark functions (Rosenbrock and Griewank) in 2, 5, and 10 dimensions, and on a synthetic hydrogeologic inverse problem involving contaminant source identification in an aquifer. Two metrics are reported: robustness (percentage of runs that locate the global optimum) and efficiency (statistical distribution of function evaluations required to reach the optimum). Across all test cases, Squads outperforms pure LM, standard PSO, APSO, and the existing hybrid hPSO (PSO + Nelder‑Mead). For the 10‑dimensional Griewank function, which contains many local minima, Squads achieves a robustness of >95 % with roughly 30‑60 % fewer function evaluations than the next best method. In the hydrogeologic case, where each forward model run costs 1–2 hours, Squads reaches the global optimum with an average of ~1,200 evaluations, compared to ~2,800 for hPSO and >4,500 for LM alone, representing a substantial reduction in computational expense.

The authors discuss limitations: LM requires gradient and curvature information, which may be expensive or unavailable for some black‑box simulators; the adaptive tribe management can increase memory usage in very high‑dimensional problems (>50 D); and the stochastic nature of the adaptive rules may lead to variability in performance. Future work is suggested on integrating automatic dimensionality reduction, parallel LM implementations, and testing on real‑world large‑scale models.

In summary, Squads delivers a robust, efficient, and largely parameter‑free optimization strategy that synergistically combines the global search capability of APSO with the rapid local convergence of LM, making it especially suitable for inverse problems where model evaluations are costly and objective landscapes are highly multimodal.