Sequential search based on kriging: convergence analysis of some algorithms

Let $\FF$ be a set of real-valued functions on a set $\XX$ and let $S:\FF \to \GG$ be an arbitrary mapping. We consider the problem of making inference about $S(f)$, with $f\in\FF$ unknown, from a finite set of pointwise evaluations of $f$. We are mainly interested in the problems of approximation and optimization. In this article, we make a brief review of results concerning average error bounds of Bayesian search methods that use a random process prior about $f$.

💡 Research Summary

The paper “Sequential search based on kriging: convergence analysis of some algorithms” investigates the theoretical performance of sequential search methods that rely on kriging (Gaussian process) priors, focusing on average‑case error bounds rather than the traditional worst‑case analysis. The authors first formalize the problem: a set F of real‑valued functions on a domain X, a mapping S: F → G, and a deterministic algorithm that, after n pointwise evaluations of an unknown function f ∈ F, produces an estimate bₙ(f) of S(f). They argue that worst‑case analysis cannot justify adaptive sampling, prompting a Bayesian viewpoint where f is modeled as a sample path of a random process ξ defined on a probability space.

The process ξ is assumed to be a zero‑mean Gaussian process with a continuous covariance function k(x,y)=Φ(x−y). The Fourier transform \tildeΦ(u) is required to satisfy two‑sided bounds of order (1+‖u‖²)^{-s} with s>d/2. This condition is satisfied, for example, by Matérn covariances with smoothness parameter ν=s−d/2. The kriging predictor bₙ(x) is the orthogonal projection of ξ(x) onto the span of the observed values ξ(X₁),…,ξ(Xₙ) in L², and its mean‑square error (MSE) is denoted σₙ²(x).

The paper separates the analysis into two tasks: function approximation and global optimization.

Function Approximation
For approximation, the authors consider the operators S(ξ)=ξ|_X and bₙ(ξ)=bₙ|_X, and evaluate designs using the maximum MSE (MMSE) and the integrated MSE (IMSE). Proposition 2 shows that, under Gaussianity, adaptive designs cannot improve the average MMSE or IMSE compared with the best non‑adaptive design. The proof relies on the fact that for any adaptive strategy the MSE at a point x is the expectation of the MSE of a corresponding non‑adaptive design with the same evaluation points, implying that the adaptive MSE is at least as large as the minimum non‑adaptive MSE.

Proposition 3 links the MMSE to the worst‑case L^∞ error over the unit ball H₁ of the reproducing kernel Hilbert space (RKHS) H generated by k. Consequently, optimal rates for MMSE and IMSE can be derived from approximation theory in Sobolev spaces. Assuming the regularity condition on k, Proposition 4 establishes that for any non‑adaptive design the IMSE and MMSE decay as C n^{-2ν/d}. Moreover, when X has a Lipschitz boundary and satisfies an interior cone condition, the same rate is achievable by space‑filling designs with fill distance hₙ≈n^{-1/d}.

Since finding the exact MMSE‑optimal design is a non‑convex problem, the authors discuss a greedy sequential strategy that at each step selects the point maximizing the current predictive variance (equation (6)). Using results from Binev et al. (2010), Proposition 5 proves that this greedy algorithm preserves the polynomial decay rate O(n^{-2ν/d}) for the MMSE, making it a practical, rate‑optimal alternative.

Global Optimization
For optimization, the target functional is S(ξ)=sup_{x∈X} ξ(x) and the estimator is the maximum observed value bₙ(ξ)=max_{i≤n} ξ(X_i). The average error ε_opt(Xₙ)=E