A Kernel-Based Approach for Modelling Gaussian Processes with Functional Information

Gaussian processes (GPs) are ubiquitous tools for modeling and predicting continuous processes in physical and engineering sciences. This is partly due to the fact that one may employ a Gaussian process as an interpolator while facilitating straightforward uncertainty quantification at other locations. In addition to training data, it is sometimes the case that available information is not in the form of a finite collection of points. For example, boundary value problems contain information on the boundary of a domain, or underlying physics lead to known behavior on an entire uncountable subset of the domain of interest. While an approximation to such known information may be obtained via pseudo-training points in the known subset, such a procedure is ad hoc with little guidance on the number of points to use, nor the behavior as the number of pseudo-observations grows large. We propose and construct Gaussian processes that unify, via reproducing kernel Hilbert space, the typical finite training data case with the case of having uncountable information by exploiting the equivalence of conditional expectation and orthogonal projections in Hilbert space. We show existence of the proposed process and establish that it is the limit of a conventional GP conditioned on an increasing number of training points. We illustrate the flexibility and advantages of our proposed approach via numerical experiments.

💡 Research Summary

The paper addresses a fundamental limitation of standard Gaussian process (GP) models: they can only incorporate information that is available at a finite set of input locations. In many scientific and engineering problems, however, one possesses exact knowledge of the process on an uncountable subset of the input domain—for example, Dirichlet boundary conditions, known physical laws, or regions where the output is analytically determined. A common workaround is to generate “pseudo‑observations” on that subset and treat them as ordinary training points, but this ad‑hoc approach leaves open the questions of how many points to use, where to place them, and how the resulting model behaves as the number of pseudo‑points grows.

The authors propose a principled construction that directly embeds such functional information into the GP. The key insight is to view conditioning on a set of observations as an orthogonal projection in a reproducing kernel Hilbert space (RKHS). Starting from a continuous, positive‑definite kernel (k) defined on a compact domain (T\subset\mathbb{R}^d), they consider the associated integral operator (K) on (L^2(T)) and its square‑root (K^{1/2}). The RKHS (H(T)) generated by (k) (which coincides with the Cameron‑Martin space) is the closure of the span of kernel sections (k(\cdot,t)) under the inner product (\langle\cdot,\cdot\rangle_{H}). In this space every evaluation functional is bounded, allowing the representation (\varrho_t(f)=\langle f,k_t\rangle_H).

Given a compact subset (T_0\subset T) on which the process values are known exactly (denoted (g_0)), the authors define a projection operator (P_{T_0}:H(T)\to H(T)) that maps any function to the unique element of (H(T)) that agrees with it on (T_0) and has minimal RKHS norm. This operator can be expressed in terms of (K) and its restriction to (T_0), leading to explicit formulas for the conditional mean and covariance: \