Measure Theory of Conditionally Independent Random Function Evaluation

In sequential design strategies, common in geostatistics and Bayesian optimization, the selection of a new observation point $X_{n+1}$ of a random function $\mathbf f$ is informed by past data, captured by the filtration $\mathcal F_n=σ(\mathbf f(X_0),\dots,\mathbf f(X_n))$. The random nature of $X_{n+1}$ introduces measure-theoretic subtleties in deriving the conditional distribution $\mathbb P(\mathbf f(X_{n+1})\in A \mid \mathcal F_n)$. Practitioners often resort to a heuristic: treating $X_0,\dots, X_{n+1}$ as fixed parameters within the conditional probability calculation. This paper investigates the mathematical validity of this widespread practice. We construct a counterexample to prove that this approach is, in general, incorrect. We also establish our central positive result: for continuous Gaussian random functions and their canonical conditional distribution, the heuristic is sound. This provides a rigorous justification for a foundational technique in Bayesian optimization and spatial statistics. We further extend our analysis to include settings with noisy evaluations and to cases where $X_{n+1}$ is not adapted to $\mathcal F_n$ but is conditionally independent of $\mathbf f$ given the filtration.

💡 Research Summary

The paper addresses a subtle but important measure‑theoretic issue that arises in sequential design strategies such as Bayesian optimization and geostatistics. In these settings a random function (f) is observed at a sequence of points (X_0,X_1,\dots). After (n) observations the available information is the σ‑algebra (\mathcal F_n=\sigma\bigl(f(X_0),\dots,f(X_n)\bigr)). Practitioners often compute the conditional distribution of the next observation (\mathbb P\bigl(f(X_{n+1})\in A\mid\mathcal F_n\bigr)) by pretending that the locations (X_0,\dots,X_{n+1}) are deterministic parameters and then plugging the random (X_{n+1}) into the resulting formula. The authors ask whether this “plug‑in” heuristic is mathematically sound.

The paper first formalises the problem. For each deterministic location (x) let (\kappa_x(\omega;A)=\mathbb P\bigl(f(x)\in A\mid\mathcal F\bigr)(\omega)) be a regular conditional distribution. The collection ({\kappa_x}_{x\in X}) is said to admit a joint kernel (\kappa(\omega,x;A)) if (i) (\kappa(\omega,x;A)=\kappa_x(\omega;A)) for all (x) and (ii) the map ((\omega,x)\mapsto\kappa(\omega,x;A)) is measurable for every Borel set (A). A joint kernel is called plug‑in consistent (PIC) if for every (\mathcal F)‑measurable random location (X) we have
\