Application of Predictive Model Selection to Coupled Models

A predictive Bayesian model selection approach is presented to discriminate coupled models used to predict an unobserved quantity of interest (QoI). The need for accurate predictions arises in a variety of critical applications such as climate, aerospace and defense. A model problem is introduced to study the prediction yielded by the coupling of two physics/sub-components. For each single physics domain, a set of model classes and a set of sensor observations are available. A goal-oriented algorithm using a predictive approach to Bayesian model selection is then used to select the combination of single physics models that best predict the QoI. It is shown that the best coupled model for prediction is the one that provides the most robust predictive distribution for the QoI.

💡 Research Summary

The paper introduces a predictive Bayesian model‑selection framework specifically designed for coupled (multiphysics) models where the quantity of interest (QoI) cannot be directly observed. Traditional Bayesian model selection relies on posterior model probabilities or marginal likelihoods (evidence) to pick the most plausible model. While this approach balances model complexity against data fit, it does not incorporate any information about the QoI, which can lead to sub‑optimal choices when multiple QoIs are relevant or when the selected model does not predict the QoI well.

To address this gap, the authors define a utility function for each candidate model that measures the negative Kullback‑Leibler (KL) divergence between the true QoI distribution (assumed to be generated by some model in the candidate set) and the model’s predictive distribution. The expected utility, taken over the posterior distribution of model parameters and over model posterior probabilities, yields a criterion that combines three sources of information: (1) the usual Bayesian evidence (through the posterior model probabilities), (2) the model‑specific predictive risk (the KL term that penalizes the loss incurred if the wrong model is chosen), and (3) the intrinsic model complexity (embedded in the posterior). By algebraic manipulation, the expected utility reduces to minimizing the KL divergence between the “global” predictive distribution (the mixture of all models weighted by their posterior probabilities) and the predictive distribution of each individual model. In other words, the best model is the one whose predictive density most closely approximates the robust, mixture‑based predictive density that aggregates all available model knowledge.

The methodology is illustrated on a synthetic coupled system consisting of a spring‑mass‑damper dynamics (physics A) and an external forcing function (physics B). Three alternative spring models (linear, cubic, quintic) and three forcing models (simple exponential decay, oscillatory linear decay, oscillatory exponential decay) generate nine possible coupled models. The true system is the quintic spring with an oscillatory exponential decay forcing, with parameters chosen to produce a non‑trivial response. The QoI is defined as the maximum absolute velocity over time, a quantity that cannot be measured directly in the simulated experiments. Observations consist of kinetic‑energy time series for the spring‑mass subsystem and force‑time data for the forcing subsystem, both corrupted with log‑normal noise.

Calibration of each candidate model’s parameters is performed using a Hybrid Gibbs Transitional Markov Chain Monte Carlo algorithm (implemented in the QUESO library), which also provides accurate estimates of the marginal likelihood via adaptive thermodynamic integration. To evaluate the KL‑based utility, the authors employ k‑nearest‑neighbor estimators that operate directly on posterior samples, avoiding the need for analytic density forms.

Three experimental scenarios are examined: (i) using the full set of nine models, (ii) restricting the candidate set to a subset that excludes the true model, and (iii) varying the amount of observational information (strong vs. weak data). In all cases, two selection strategies are compared: (a) conventional Bayesian model selection based on the highest posterior probability (or evidence), and (b) the proposed predictive selection based on maximizing expected utility (i.e., minimizing the KL divergence to the mixture predictive distribution).

Results show that when data are informative enough to discriminate models, both methods tend to select the true model. However, under limited or ambiguous data, the conventional approach often defaults to simpler models (e.g., the linear spring with simple exponential decay) because they achieve higher marginal likelihoods despite providing poor QoI predictions. In contrast, the predictive selection consistently identifies the model whose predictive distribution is closest to the mixture distribution, which in the experiments corresponds to the true quintic‑oscillatory‑exponential model or, at worst, a model that yields a more accurate estimate of the maximum velocity. Quantitatively, the predictive‑selected models achieve lower QoI prediction errors and exhibit reduced sensitivity to parameter uncertainty.

The paper’s contributions are threefold: (1) formal integration of QoI‑specific information into Bayesian model selection via a KL‑based utility, (2) derivation of an intuitive criterion that selects the model whose predictive density best approximates the robust mixture predictive density, and (3) demonstration of a practical computational pipeline that combines advanced MCMC sampling with sample‑based KL estimators. The authors also discuss the connection between predictive selection and optimal experimental design, suggesting that experiments can be planned to maximize the discriminative power of the QoI‑aware utility.

In conclusion, predictive Bayesian model selection offers a principled way to choose coupled models that are most reliable for the target QoI, especially when data are scarce or when multiple competing models have similar evidence. The framework is readily extensible to higher‑dimensional multiphysics problems, to multiple QoIs, and to sequential decision‑making contexts where model updates and new data acquisition are interleaved. Future work may explore automated design of sensor placements, incorporation of hierarchical model structures, and scalability to large‑scale engineering simulations.