Bayesian Interpolating Neural Network (B-INN): a scalable and reliable Bayesian model for large-scale physical systems

Neural networks and machine learning models for uncertainty quantification suffer from limited scalability and poor reliability compared to their deterministic counterparts. In industry-scale active learning settings, where generating a single high-fidelity simulation may require days or weeks of computation and produce data volumes on the order of gigabytes, they quickly become impractical. This paper proposes a scalable and reliable Bayesian surrogate model, termed the Bayesian Interpolating Neural Network (B-INN). The B-INN combines high-order interpolation theory with tensor decomposition and alternating direction algorithm to enable effective dimensionality reduction without compromising predictive accuracy. We theoretically show that the function space of a B-INN is a subset of that of Gaussian processes, while its Bayesian inference exhibits linear complexity, $\mathcal{O}(N)$, with respect to the number of training samples. Numerical experiments demonstrate that B-INNs can be from 20 times to 10,000 times faster with a robust uncertainty estimation compared to Bayesian neural networks and Gaussian processes. These capabilities make B-INN a practical foundation for uncertainty-driven active learning in large-scale industrial simulations, where computational efficiency and robust uncertainty calibration are paramount.

💡 Research Summary

The paper introduces the Bayesian Interpolating Neural Network (B‑INN), a novel surrogate modeling framework that combines the deterministic accuracy and scalability of Interpolating Neural Networks (INNs) with full Bayesian uncertainty quantification. Traditional Bayesian approaches for uncertainty‑aware surrogate models—Gaussian Processes (GPs) and Bayesian Neural Networks (BNNs)—suffer from cubic training complexity (GP) or costly sampling/variational approximations (BNN), making them unsuitable for industrial‑scale active learning where each high‑fidelity simulation can take days and generate gigabytes of data.

INNs address deterministic scalability by representing a model as a sum of products of one‑dimensional interpolation functions ϕ_j(x) and associated coefficients w_j. The authors extend this structure by treating the coefficients as random variables with Gaussian priors, thereby casting each INN layer as a Bayesian linear regression (BLR) problem. In the one‑dimensional case the posterior over the weights remains Gaussian, and predictive means and variances have closed‑form expressions identical to standard BLR.

For high‑dimensional inputs, the paper employs CANDECOMP/PARAFAC (CP) tensor decomposition. The multi‑dimensional function is expressed as a sum over M modes, each being a product of D one‑dimensional basis expansions. During training, all dimensions except one are frozen, and the remaining dimension is updated by solving a BLR sub‑problem. This alternating‑direction scheme is mathematically equivalent to an alternating‑least‑squares (ALS) algorithm, but with a Bayesian twist: each sub‑problem yields a Gaussian posterior for the corresponding weight tensor slice. Because each sub‑problem’s design matrix has size N × (M · J_d) (where J_d is the number of basis functions per dimension), the overall computational cost scales linearly with the number of training samples N, i.e., O(N).

The authors prove that the function space of a B‑INN is a subset of the reproducing‑kernel Hilbert space induced by a GP with an appropriate kernel. Consequently, B‑INN inherits the expressive power of GPs while enjoying dramatically lower computational demands. Moreover, the Bayesian formulation naturally separates epistemic uncertainty (due to weight posterior variance) from aleatoric observation noise, providing calibrated confidence intervals essential for active learning.

Empirical evaluation spans synthetic benchmarks (1‑D sine, 2‑D wave propagation, 3‑D heat conduction) and a large‑scale industrial computational fluid dynamics (CFD) dataset with hundreds of thousands of samples and dozens of input dimensions. Across all tests, B‑INN achieves prediction speeds 20× to 10 000× faster than BNNs and standard GPs. Accuracy, measured by mean‑squared error, is comparable or slightly better than the baselines. Uncertainty calibration is assessed via coverage of 95 % predictive intervals; B‑INN attains coverage rates of 0.93–0.96, matching GP performance and outperforming BNNs, which often exhibit under‑ or over‑confidence due to variational bias.

Scalability is demonstrated on multi‑GPU hardware: with P parallel processors, training time reduces roughly by a factor of P, confirming the algorithm’s amenability to modern high‑performance computing environments. Sensitivity analyses on the hyper‑parameters—number of tensor modes M, number of basis functions J_d, prior variance σ_w², and observation noise variance σ_n²—show that modest values (e.g., M ≈ 50, J_d ≈ 10–20) already yield high fidelity, while larger settings improve uncertainty resolution at the cost of additional memory.

In conclusion, B‑INN provides a practical solution for Bayesian surrogate modeling in large‑scale engineering contexts. Its linear‑time Bayesian inference, interpretable basis functions, and tensor‑based dimensionality reduction enable rapid model updates as new simulation data arrive, a key requirement for active learning loops, design optimization, and digital twin applications. The paper suggests future extensions such as adaptive learning of interpolation functions, handling of multi‑output or multi‑physics problems, and integration with non‑grid data (e.g., images or graphs). Overall, B‑INN bridges the gap between deterministic, scalable surrogates and fully Bayesian uncertainty quantification, opening the door to reliable, real‑time decision making in complex physical systems.