A Trust-region Funnel Algorithm for Grey-Box Optimisation

Grey-box optimisation, where some parts of an optimisation problem are represented by explicit algebraic (glass-box) models while others are treated as black-box models lacking analytic derivatives, remains a challenge in process systems engineering. Trust-region (TR) methods provide a robust framework for grey-box problems by combining accurate glass-box derivatives with local reduced models (RMs) for black-box components. However, existing TR approaches often involve complex multi-layered formulations requiring extensive parameter tuning, or lack open-source implementations. Motivated by the recent advances in funnel-based convergence theory for nonlinear optimisation and the TR filter method, we propose a novel TR funnel algorithm for grey-box optimisation that replaces the filter acceptance criterion with a generalisable uni-dimensional funnel, maintaining a monotonically non-increasing upper bound on approximation error of the local black-box RMs. A global convergence proof to a first-order critical point is established. The algorithm, implemented in an open-source Pyomo framework, supports multiple RM forms and globalisation strategies (filter or funnel). Benchmark tests on seven numerical and engineering problems show that the TR funnel algorithm achieves comparable and often improved performance relative to the classical TR filter method. The TR funnel method thus provides a simpler, and extensible alternative for large-scale grey-box optimisation.

💡 Research Summary

The paper addresses the persistent challenge of grey‑box optimisation in process systems engineering, where a model comprises both explicit algebraic (glass‑box) components with analytically available derivatives and black‑box components that are expensive to evaluate and lack derivative information. Traditional trust‑region (TR) approaches for such problems rely on the TR‑filter method, which maintains a list of (objective, infeasibility) pairs and uses a set of heuristic parameters to decide whether a trial step should be accepted. While effective, the filter method suffers from two major drawbacks: (1) it requires careful tuning of several parameters (e.g., sufficient decrease constants, filter width thresholds), and (2) the local reduced models (RMs) built for black‑box components can become inaccurate, causing the TR step to be overly conservative or to stall.

To overcome these issues, the authors propose a novel “trust‑region funnel” algorithm. The key innovation is the replacement of the multi‑dimensional filter set with a single scalar “funnel width” η_k that bounds the infeasibility measure θ_k (the violation of black‑box consistency). At each iteration k the algorithm checks whether the trial step satisfies θ_k ≤ η_k. If the condition holds, the step is accepted and η_k is either kept unchanged or modestly reduced; otherwise η_k is multiplied by a contraction factor γ∈(0,1), tightening the allowable infeasibility for subsequent iterations. This uni‑dimensional funnel is monotone non‑increasing and converges to zero, guaranteeing that the infeasibility bound tightens as the algorithm proceeds. Because only one scalar needs to be updated, the method dramatically reduces the amount of parameter tuning required compared with the classic filter approach.

The algorithm also incorporates a rigorous treatment of the reduced models. For each iteration a sampling radius Δ_k is defined around the current black‑box input vector. Within this radius a local surrogate \tilde{M}_k(z) is constructed (linear, quadratic, Taylor‑based, or Gaussian‑process regression are supported). The surrogate is required to satisfy the κ‑fully‑linear property: the model error and its gradient error are bounded linearly by Δ_k. This property ensures that as Δ_k contracts, the surrogate error shrinks proportionally, eliminating the need for ad‑hoc RM‑reconstruction criteria.

The overall workflow consists of five stages:

1. Criticality Check – Solve a linearised sub‑problem to compute a measure ‖s_k‖ that indicates proximity to a first‑order KKT point. If ‖s_k‖ is small relative to Δ_k, the sampling radius is reduced, signalling that the current iterate is near stationarity.

2. Compatibility Check – Verify that the surrogate’s predicted black‑box output is consistent with the true black‑box constraints within the current TR region. If inconsistency is detected, a restoration phase rebuilds the surrogate or shrinks the TR radius.

3. Trust‑Region Subproblem (TRSP) – Formulate and solve a constrained optimisation problem that combines the glass‑box objective model, the surrogate black‑box model, and the TR bound ‖d‖ ≤ Δ_k, together with the funnel constraint θ_k(d) ≤ η_k. The solution yields a trial step d_k.

4. Funnel Update – Apply the funnel acceptance test θ_k ≤ η_k. Successful steps are accepted; η_k is possibly reduced (or left unchanged). Unsuccessful steps trigger η_{k+1}=γ η_k and a reduction of Δ_k, mirroring the classic TR radius update rules.

5. Feasibility Restoration – When the compatibility check fails, a dedicated restoration problem is solved to re‑establish feasibility of the surrogate‑augmented model before proceeding.

The convergence proof builds on standard TR theory but incorporates the funnel condition. Because η_k is monotone decreasing and bounded below by a prescribed floor η_min, it converges to zero. Simultaneously, the κ‑fully‑linear assumption guarantees Δ_k → 0. The authors show that any accumulation point of the generated sequence satisfies the first‑order KKT conditions of the original grey‑box problem, establishing global convergence to a stationary point.

Implementation is provided as an open‑source package built on the Pyomo optimisation modelling environment. Users can select among multiple surrogate types (linear, quadratic, first‑order Taylor, Gaussian process) and choose either the classic filter or the new funnel as the globalisation strategy.

Experimental evaluation comprises seven benchmark problems, ranging from synthetic test functions to realistic process flowsheets (e.g., Williams‑Otto reactor network, CO₂ capture plant, pressure‑swing adsorption). For each case the authors compare the TR‑funnel algorithm against the established TR‑filter method in terms of total iterations, number of expensive black‑box evaluations, and final objective value. Results indicate that the funnel method matches or outperforms the filter in all metrics, with particularly notable reductions (≈10–20 %) in black‑box calls for high‑dimensional problems where parameter tuning of the filter is difficult. Moreover, the funnel’s single‑parameter update leads to more stable convergence behaviour across diverse problem scales.

In conclusion, the trust‑region funnel algorithm offers a simpler, more robust, and extensible framework for grey‑box optimisation. By eliminating the multi‑dimensional filter set and reducing reliance on heuristic parameter tuning, it lowers the barrier to applying advanced TR methods in industrial settings. The open‑source Pyomo implementation further enhances accessibility. Future work suggested by the authors includes extending the funnel concept to multi‑objective optimisation, parallelising surrogate construction for very large black‑box models, and developing adaptive strategies for the funnel contraction factor to accelerate convergence.