Multi-Objective Multi-Fidelity Bayesian Optimization with Causal Priors

Multi-fidelity Bayesian optimization (MFBO) accelerates the search for the global optimum of black-box functions by integrating inexpensive, low-fidelity approximations. The central task of an MFBO policy is to balance the cost-efficiency of low-fidelity proxies against their reduced accuracy to ensure effective progression toward the high-fidelity optimum. Existing MFBO methods primarily capture associational dependencies between inputs, fidelities, and objectives, rather than causal mechanisms, and can perform poorly when lower-fidelity proxies are poorly aligned with the target fidelity. We propose RESCUE (REducing Sampling cost with Causal Understanding and Estimation), a multi-objective MFBO method that incorporates causal calculus to systematically address this challenge. RESCUE learns a structural causal model capturing causal relationships between inputs, fidelities, and objectives, and uses it to construct a probabilistic multi-fidelity (MF) surrogate that encodes intervention effects. Exploiting the causal structure, we introduce a causal hypervolume knowledge-gradient acquisition strategy to select input-fidelity pairs that balance expected multi-objective improvement and cost. We show that RESCUE improves sample efficiency over state-of-the-art MF optimization methods on synthetic and real-world problems in robotics, machine learning (AutoML), and healthcare.

💡 Research Summary

The paper introduces RESCUE, a novel multi‑objective multi‑fidelity Bayesian optimization (MF‑MOBO) framework that explicitly incorporates causal knowledge. Traditional MF‑BO methods rely on statistical correlations among inputs, fidelities, and objectives, which can lead to poor performance when low‑fidelity sources are misaligned with the target fidelity. RESCUE first learns a structural causal model (SCM) from observational data using algorithms such as PC or DirectLiNGAM, yielding a directed acyclic graph that captures causal dependencies among configuration variables, key performance indicators, objectives, and fidelity levels.

From this causal graph, the authors derive an interventional expectation f_do(x, s) and an associated variance estimate bσ(x). These quantities form a causal prior: the mean of a multi‑output Gaussian process (GP) is set to f_do(x, s), while the variance term incorporates the causal uncertainty. The GP kernel is a tensor product of three components: an RBF kernel over configurations (k_in), an RBF kernel over fidelities (k_fid), and a coregionalization kernel over objectives (k_obj). This “MF‑Causal GP” (MF‑CGP) simultaneously models spatial similarity, cross‑fidelity correlation, and multi‑objective relationships, while being grounded in causal reasoning.

For acquisition, the authors extend the Hypervolume Knowledge‑Gradient (HVKG) to a Causal Hypervolume Knowledge‑Gradient (C‑HVKG). The acquisition function evaluates the expected increase in hypervolume of the Pareto front at the target fidelity, combining (i) the predictive uncertainty of the MF‑CGP, (ii) a causal hypervolume term derived from f_do, and (iii) a cost normalization factor c(x, s). A weight w balances the contribution of the data‑driven GP and the causal prior. The expectation is approximated via Monte‑Carlo sampling, and the next (x, s) pair is selected by maximizing C‑HVKG under feasibility constraints.

RESCUE iteratively updates the causal graph (every N_l iterations), retrains the MF‑CGP, and acquires new samples until a budget Λ is exhausted. The final Pareto set is extracted on the target fidelity using NSGA‑II applied to the GP posterior mean.

Theoretical analysis shows that RESCUE’s expected performance is bounded by that of a single‑fidelity MOBO, guaranteeing that the causal augmentation cannot degrade performance. Empirical evaluation spans synthetic benchmarks, robot parameter tuning in Gazebo, AutoML hyper‑parameter search, and a healthcare risk‑prediction task. Across all domains, RESCUE achieves 30‑60 % fewer evaluations than state‑of‑the‑art MF‑MOBO methods and outperforms single‑fidelity BO when low‑fidelity sources are poorly aligned. The results demonstrate that causal priors effectively guide exploration, especially in early iterations when data are scarce, leading to faster convergence to high‑quality Pareto fronts.