Semi-empirical correction of ab initio harmonic properties by scaling factors: a validated uncertainty model for calibration and prediction

Bayesian Model Calibration is used to revisit the problem of scaling factor calibration for semi-empirical correction of ab initio harmonic properties (e.g. vibrational frequencies and zero-point energies). A particular attention is devoted to the evaluation of scaling factor uncertainty, and to its effect on the accuracy of scaled properties. We argue that in most cases of interest the standard calibration model is not statistically valid, in the sense that it is not able to fit experimental calibration data within their uncertainty limits. This impairs any attempt to use the results of the standard model for uncertainty analysis and/or uncertainty propagation. We propose to include a stochastic term in the calibration model to account for model inadequacy. This new model is validated in the Bayesian Model Calibration framework. We provide explicit formulae for prediction uncertainty in typical limit cases: large and small calibration sets of data with negligible measurement uncertainty, and datasets with large measurement uncertainties.

💡 Research Summary

This paper revisits the widely used practice of applying a single scaling factor to correct ab‑initio harmonic properties such as vibrational frequencies and zero‑point energies. In the conventional approach the relationship between experimental values (ν_exp) and calculated harmonic frequencies (ν_calc) is modeled as ν_exp = s · ν_calc + ε, where s is the scaling factor and ε represents the experimental measurement error. The factor s is normally obtained by ordinary least‑squares regression, assuming that the model can reproduce the experimental data within the quoted uncertainties.

The authors demonstrate that, for most realistic data sets, this assumption is violated: the residuals after fitting are systematically larger than the experimental uncertainties, indicating that the simple linear model is statistically invalid. The source of the discrepancy is identified as model inadequacy – systematic deviations arising from the chosen electronic structure method, basis set, and the harmonic approximation itself, which are not captured by the measurement error term ε.

To address this, the paper introduces a stochastic discrepancy term δ and adopts a Bayesian Model Calibration (BMC) framework. The revised model reads

 ν_exp = s · ν_calc + δ + ε,

where δ ∼ N(0, σ_δ²) accounts for the unknown model error and ε ∼ N(0, σ_ε²) is the known experimental noise. Both s and σ_δ² are treated as random variables with weakly informative priors. Using Markov‑Chain Monte Carlo sampling, the joint posterior distribution p(s, σ_δ² | data) is obtained, allowing a rigorous quantification of the scaling factor’s uncertainty and of the model‑inadequacy variance.

The authors explore three limiting regimes. (1) Large calibration sets with negligible measurement error – the posterior for s collapses to a narrow Gaussian centred on the ordinary‑least‑squares estimate, but the posterior variance includes a contribution from σ_δ², which is often comparable to or larger than the variance inferred from residuals alone. Consequently, the conventional method underestimates the true uncertainty. (2) Small calibration sets – the posterior is dominated by the prior; uncertainty in s is substantial, and the inferred σ_δ² is poorly constrained. The paper advises caution and recommends augmenting the data set or employing more informative priors. (3) Data with sizable experimental uncertainties – σ_δ² becomes the dominant term in the predictive variance, dwarfing the contribution from s. In this regime, the predictive uncertainty for a new quantity ν_pred is given by

 Var(ν_pred) = s² Var(ν_calc) + σ_δ² + σ_ε²,

where Var(ν_calc) is typically negligible for high‑level quantum‑chemical calculations. The formula is derived explicitly for each regime, providing practitioners with ready‑to‑use expressions for uncertainty propagation to derived properties such as zero‑point energies.

The paper validates the Bayesian model on several benchmark data sets spanning different levels of theory (e.g., HF, DFT, MP2) and molecular sizes. In all cases the posterior predictive checks confirm that the model can reproduce the experimental distribution within the combined uncertainty envelope, whereas the standard scaling‑factor model fails this test.

In summary, the study shows that the traditional scaling‑factor calibration is statistically inadequate for most practical applications. By embedding a discrepancy term within a Bayesian calibration framework, the authors obtain a calibrated scaling factor together with a realistic estimate of its uncertainty and of the model‑inadequacy variance. This enables reliable uncertainty quantification and propagation in computational spectroscopy and thermochemistry, offering a robust methodological foundation for future high‑accuracy quantum‑chemical predictions.