---

APPEARED IN BULLETIN OF THE

arXiv:math/9307234v1 [math.NA] 1 Jul 1993APPEARED IN BULLETIN OF THEAMERICAN MATHEMATICAL SOCIETYVolume 29, Number 1, July 1993, Pages 70-76AVERAGE CASE COMPLEXITYOF LINEAR MULTIVARIATE PROBLEMSH. Wo´zniakowskiAbstract.

We study the average case complexity of a linear multivariate problem(LMP) defined on functions of d variables. We consider two classes of information.The first Λstd consists of function values and the second Λall of all continuous linearfunctionals.

Tractability of LMP means that the average case complexity is O((1/ε)p)with p independent of d. We prove that tractability of an LMP in Λstd is equivalentto tractability in Λall, although the proof is not constructive. We provide a simplecondition to check tractability in Λall.We also address the optimal design problem for an LMP by using a relation tothe worst case setting.

We find the order of the average case complexity and optimalsample points for multivariate function approximation. The theoretical results areillustrated for the folded Wiener sheet measure.1.

IntroductionA linear multivariate problem (LMP) is defined as the approximation of a con-tinuous linear operator on functions of d variables. Many LMP’s are intractablein the worst case setting.

That is, the worst case complexity of computing an ε-approximation is infinite or grows exponentially with d (see, e.g., [9]). For example,consider multivariate integration and function approximation of r times continu-ously differentiable functions of d variables.

Then the worst case complexity is oforder (1/ε)d/r assuming that an ε-approximation is computed using function val-ues. Thus, if only continuity of the functions is assumed, r = 0, then the worstcase complexity is infinite.

For positive r, if d is large relative to r, then the worstcase complexity is huge even for modest ε. In either case, the problem cannot besolved.In this paper we study if tractability can be broken by replacing the worst casesetting by an average case setting with a Gaussian measure on the space of functions.The average case complexity is defined as the minimal average cost of computing anapproximation with average error at most ε.

We consider two classes of information.The first class Λstd consists of function values, and the second class Λall consists ofall continuous linear functionals.We say an LMP is tractable if the average case complexity is O ((1/ε)p) with pindependent of d. The smallest such p is called the exponent of the problem. Undermild assumptions, we prove that tractability in Λall is equivalent to tractability in1991 Mathematics Subject Classification.

Primary 68Q25; Secondary 65D15, 41A65.Received by the editors January 6, 1992This research was supported in part by the National Science Foundation under Contract IRI-89-07215 and by AFOSR-91-0347c⃝1993 American Mathematical Society0273-0979/93 $1.00 + $.25 per page1

2H. WO´ZNIAKOWSKIΛstd and that the difference of the exponents is at most 2.

The proof of this resultis not constructive. We provide, however, a simple condition to check tractabilityin Λall.In particular, this means that multivariate integration is tractable in Λstd and itsexponent is at most 2.

This should be contrasted with the worst case setting where,even for d = 1, the worst case complexity in Λstd can be infinite or an arbitraryincreasing function of 1/ε (see [14]). Of course, intractability of multivariate inte-gration in the worst case setting can also be broken by switching to the randomizedsetting and using the classical Monte Carlo algorithm.The optimal design problem of constructing sample points which achieve (ornearly achieve) the average case complexity of an LMP in Λstd is a challengingproblem.

This problem has long been open even for multivariate integration andfunction approximation. In what follows, we will use the word “optimal” modulo amultiplicative constant which may depend on d but is independent of ε. Recently,the optimal design problem has been solved for multivariate integration for specificGaussian measures (see [15] for the classical Wiener sheet measure, [5] for the thefolded Wiener sheet measure, and [13] for the isotropic Wiener measure).In this paper, we show under a mild assumption that tractability of functionapproximation (APP) implies tractability of other LMPs.

Therefore, it is enoughto address optimal sample points for APP. Optimal design for APP is analyzed byexhibiting a relation between average case and worst case errors of linear algorithmsfor APP.

This relation reduces the study of the average case to the worst case fora different class of functions. This different class is the unit ball of a reproducingkernel Hilbert space whose kernel is given by the covariance kernel of the averagecase measure.

Similar relations have been used in many papers for approximatingcontinuous linear functionals; a thorough overview may be found in [11].We illustrate the theoretical results for the folded Wiener sheet measure. In thiscase, an LMP is tractable and has exponent at most 2.

For APP the exponents inΛstd and Λall are the same. The exponent in Λall was known (see [4]), whereas theexponent in Λstd was known to be at most 6 (see [3]).

Tractability of APP for thefolded Wiener sheet measure is in sharp contrast to intractability of APP for theisotropic Wiener measure; see [13].Tractability of APP in the average case setting is significant, since it is knownthat the randomized setting does not help (see [12]). Thus, unlike multivariateintegration, intractability of APP in the worst case setting cannot be broken by therandomized setting.APP has been studied in Λstd for d = 1 in [2, 6].

For d ≥1, it was shown in[4] that the number of grid points needed to guarantee an average error ε dependsexponentially on d. Of course, O(ε−2−δ) sample points are enough to compute anε-approximation, δ > 0. Hence, grid points are a poor choice of sample points.In [4], the average case complexity of APP in Λall was found, and it was conjec-tured that the average case complexity in Λstd is of the same order.

We prove thatthis is indeed the case.Optimal design for APP is solved by using a relation to the worst case setting inthe reproducing kernel Hilbert space H. For the folded Wiener sheet measure, His a Sobolev space of smooth nonperiodic functions which satisfy certain boundaryconditions.APP in the worst case setting has been studied in this Sobolev space additionallyassuming periodicity of functions in [7, 8] (see also [10] for d = 2). It was proven

AVERAGE CASE COMPLEXITY OF LINEAR MULTIVARIATE PROBLEMS3that hyperbolic cross points are optimal sample points. Hyperbolic cross points aredefined as a subset of grid points whose indices satisfy a “hyperbolic” inequality.Approximation of periodic functions by trigonometric polynomials that use Fouriercoefficients with these hyperbolic cross indices was first studied in [1].For the nonperiodic case, optimal sample points for APP in the average casesetting are derived from hyperbolic cross points, and the average case complexityis given bycompavg(ε; APP) = Θε−1/(rmin+1/2)log 1/ε(k∗−1)(rmin+1)/(rmin+1/2),with rmin = min1≤i≤d ri, where f (r1,...,rd) is continuous and where k∗denotes thenumber of ri equal to rmin.

An optimal algorithm is given by a linear combinationof function values at sample points derived from hyperbolic cross points.Proofs of the results reported here can be found in [16].2. Linear multivariate problemsA linear multivariate problem LMP = {LMPd} is a sequence of LMPd = (F, µ,G, S, Λ) may depend on d. We now define them in turn.Let F be a separable Banach space of functions f : D →R, F ⊂L2(D).Here, D ⊂Rd, and its Lebesgue volume l(D) is in (0, +∞).

We assume that allL(f) = f(x) are in F ∗.The space F is equipped with a zero mean Gaussian measure µ. Let Rµ be thecovariance kernel of µ, i.e., Rµ(t, x) =RF f(t) f(x) µ (df) for t, x ∈D.Let S : F →G be a continuous linear operator, where G is a separable Hilbertspace.

Then ν = µS−1 is a zero mean Gaussian measure on the Hilbert space G.Its covariance operator Cν = C∗ν ≥0 and has a finite trace.Finally, Λ is either Λall = F ∗or Λstd which consists of L(f) = f(x), ∀f ∈F, forx ∈D.Our aim is to approximate elements S(f) by U(f).The latter is defined asfollows. Information about f is gathered by computing a number of L(f), whereL ∈Λ,N(f) = [L1(f), L2(f), .

. .

, Ln(f)],∀f ∈F.The choice of Li and n = n(f) may depend adaptively on the already computedinformation (see [9, Chapter 3]). Knowing y = N(f), we compute U(f) = φ(y) forsome φ : N(F) →G.

The average error of U is defined aseavg(U) =ZF∥S(f) −U(f)∥2 µ (df)1/2.To define the average cost of U, assume that each evaluation of L(f), L ∈Λ andf ∈F, costs c = c(d) > 0. Assume that we can perform arithmetic operationsand comparisons on real numbers as well as addition of two elements from G andmultiplying an element from G by a scalar; all of them with cost taken as unity.Usually c ≫1.For U(f) = φ(N(f)), let cost(N, f) denote the information cost of computingy = N(f).

Clearly, we have cost(N, f) ≥cn(f). Let n1(f) denote the number

4H. WO´ZNIAKOWSKIof operations needed to compute φ(y) given y = N(f).

(It may happen thatn1(f) = +∞.) The average cost of U is then given ascostavg(U) =ZF( cost(N, f) + n1(f) ) µ (df).The average case complexity of LMPd is the minimal cost of computing ε-approxi-mations,compavg(ε; LMPd) = inf{costavg(U) : U such that eavg(U) ≤ε}.To stress the dependence on certain parameters in compavg(ε; LMPd), we will some-times list only those.

Obviously, compavg(ε; d, Λall) ≤compavg(ε; d, Λstd). We showthat the average case complexity functions in Λall and Λstd are usually closely re-lated.3.

Tractability of linear multivariate problemsAn LMP = {LMPd} is called tractable if there exists p ≥0 such that for all d(3.1)compavg(ε; LMPd) = Oc ε−p.The constant in the big O notation may depend on d. The infimum of the numbersp satisfying (3.1) is called the exponent p∗= p∗(LMP). To stress the role of theclass Λ, we say that an LMP is tractable in Λ iff(3.1) holds for Λ.In what follows, by multivariate function approximation we mean APP = LMPwith the embedding S(f) = Id(f) = f ∈G = L2(D), where the norm in L2(D) isdenoted by ∥· ∥d.We assume that for all d there exist Ki = Ki(d), i = 1, 2, such that∥S(f)∥≤K1 ∥f∥d,∀f ∈F,(A.1)∥Rµ(·, ·)∥L∞(D) ≤K2.

(A.2)Theorem 3.1. Suppose (A.1) and (A.2) hold.

(i) Tractability of LMP in Λstd is equivalent to tractability of LMP in Λall sincecompavg(ε; d, Λall) = O(c ε−p(d)) implies compavg(ε; d, Λstd) = O(c ε−p(d)−2). (ii) Let λi(d) be the ordered eigenvalues of the covariance operator of µS−1.

LMPis tractable in Λall iffthere exists a positive number α such that for all d,(3.2)+∞Xi=n+1λi(d) = O(n−2α),as n →+∞.The exponent of LMP is p∗= 1/ sup{α : α of (3.2)}, and p∗= +∞if there is nosuch α. (iii) Tractability of APP in Λ with exponent p∗implies tractability of an LMPin Λ with exponent at most p∗provided LMP differs from APP only by the choiceof S.We stress that the proof of Theorem 3.1 is not constructive.

The exponents inΛall and Λstd may differ by at most 2. The constant 2 is sharp.

Indeed, for theintegration problem with the isotropic Wiener measure, the exponent in Λstd is 2(see [13]), and, obviously, the exponent in Λall is zero.

AVERAGE CASE COMPLEXITY OF LINEAR MULTIVARIATE PROBLEMS54. Relation to worst caseDue to (iii) of Theorem 3.1, it is enough to analyze multivariate function ap-proximation APP = {APPd} with APPd = {F, µ, L2(D), Id, Λstd}.The aver-age case errors of APP are related to worst case errors of the same Id restrictedto a specific subset of F.This specific subset of F is the unit ball BHµ of areproducing kernel Hilbert space Hµ.The space Hµ is the completion of finite-dimensional spaces of the formspan (Rµ(·, x1), Rµ(·, x2), .

. .

, Rµ(·, xk)) .The completion is with respect to ∥·∥µ = ⟨·, ·⟩1/2µ , where ⟨R(·, x), R(·, t)⟩µ = R(x, t).Consider a linear U which uses sample points xj. That is, we have U(f) =Pnj=1 f(xj) gj, where gj ∈L∞(D).

It is easy to show thateavg(U) = eavg(U; APPd) =ZD∥h∗(·, x)∥2µ dx1/2,where h∗(·, x) = Rµ(·, x) −Pnj=1 gj(x) Rµ(·, x) ∈Hµ.Consider now the same U for multivariate function approximation in the L∞(D)normAPPword= {BHµ, L∞(D), Id, Λstd}in the worst case setting. We now assume that Hµ is a subset of L∞(D) and thatthe embedding Id maps Hµ into L∞(D).

The worst error of U is equal toewor(U; APPword) = supf −U(f)L∞(D) : ∥f∥µ ≤1.It is easy to show that ewor(U; APPword) = ess supx∈D ∥h∗(·, x)∥µ, which yields(4.1)eavg(U; APPd) ≤pl(D) ewor(U; APPword),where l(D) is the Lebesgue volume of D.5. Application for folded Wiener sheet measuresWe assume that µ is the folded Wiener sheet measure (see [4]).

That is, D =[0, 1]d and F is the space of ri times continuously differentiable functions withrespect to xi which vanish with their derivatives at points with at least one com-ponent equal to zero. The norm of F is the sup norm on (r1, .

. .

, rd) derivatives.The covariance kernel Rµ of µ isRµ(t, x) =dYj=1Z 10(tj −s)rj+rj! (xj −s)rj+rj!ds.Observe that Rµ(t, t) ≤1 and (A.2) holds with K2 ≤1.The space Hµ consists now of functions f of the form (see [5])f(x) =ZDdYj=1(xj −tj)rj+rj!φ(t1, t2, .

. .

, td) dt1 dt2 · · · dtd,∀x∈D, φ∈L2(D).

6H. WO´ZNIAKOWSKIThe inner product of Hµ is ⟨f, g⟩µ =RD f (r1,...,rd)(t) g(r1,...,rd)(t) dt.Average case errors for APPd can be bounded (see (4.1)) by analyzing the worstcase ofAPPword= {BHµ, L∞(D), Id, Λstd}.Let W0 be a subspace of Hµ of periodic functions for which f (i1,...,id) (t) = 0 forall ij ≤rj and all t from the boundary of D. Multivariate function approximationfor the unit ball of W0 in the worst case setting has been analyzed by Temlyakov in[7, 8].

He constructed sample points xj and functions aj such that for Tn(f, x) =Pnj=1 f(xj) aj(x) we have(5.1)∥f −Tn(f, ·)∥L∞(D) = O(n−(rmin+1/2) (log n)(k∗−1)(rmin+1)),where rmin = min{rj : 1 ≤j ≤d} and k∗= card({j : rj = rmin}).The sample points xj are called hyperbolic cross points and the functions aj areobtained by linear combinations of the de la Vall´ee-Poussin kernel.To extend Temlyakov’s result to nonperiodic functions, define for f from BHµg(x) = f⃗h(x),∀x ∈D,where ⃗h(x) =h(x1), h(x2), . .

. , h(xd)and h(u) = 4 u (1 −u), ∀u ∈[0, 1].Observe that g is periodic and enjoys the same smoothness as f; that is, g ∈W0.There exists a constant K = K(d,⃗r) such that ∥g∥µ ≤K.

DefineU ∗n(f, t) = Tn(g,⃗h−1(t)),where ⃗h−1(t) = ( 12(1 −√1 −t1), . .

. , 12(1 −√1 −td)), t ∈D.

We have(5.2)U ∗n(f, t) =nXj=1f⃗h(xj)aj⃗h−1(t)=nXj=1f(x∗j)h∗j(t),where x∗j = ⃗h(xj), with a hyperbolic cross point xj, and h∗j(t) = aj⃗h−1(t).It is possible to check that for all f from BHµ we have(5.3)∥f −U ∗n(f, ·)∥L∞(D) = O(n−(rmin+1/2) (log n)(k∗−1)(rmin+1)).From (5.3) and (4.1) we conclude that(5.4)compavg(ε; APPd) = O(c ε−1/(rmin+1/2)(log 1/ε)(k∗−1)(rmin+1)/(rmin+1/2)).Clearly, compavg(ε; APPd) is bounded from below by the corresponding averagecase complexity in the class Λall. The latter was determined in [4].

These twoaverage case complexity functions differ by at most a constant. Thus, the O in(5.4) can be replaced by Θ.

Furthermore, the linear approximation U ∗n given by(5.2) is optimal, i.e., U ∗n computes an ε-approximation with the average cost (c+2)nwhich is minimal, modulo a constant, if(5.5)n = O(ε−1/(rmin+1/2) (log 1/ε)(k∗−1)(rmin+1)/(rmin+1/2)).

AVERAGE CASE COMPLEXITY OF LINEAR MULTIVARIATE PROBLEMS7Theorem 5.1. For APPthe average case complexity functions compavg(ε; d, Λstd)and compavg(ε; d, Λall) differ at most by a constant andcompavg(ε; d, Λstd) = Θ(c ε−1/(rmin+1/2) (log 1/ε)(k∗−1)(rmin+1)/(rmin+1/2)).The linear U ∗n given by (5.2) which uses n sample points derived from the hyperboliccross points with n given by (5.5) is optimal in the classes Λstd and Λall.From Theorem 5.1 we have that APP is tractable in Λstd since 1/(rmin+1/2) ≤2.The exponent of APP is the same in Λall and Λstd.

Since ri may depend on d, wehavep∗(Λstd) = (1/2 + min{rj(d) : j = 1, 2, . .

., d and d = 1, 2, . .

. })−1 ≤2.Obviously, any LMP which satisfies (A.1) and which is equipped with the foldedWiener sheet measure is tractable and has exponent at most p∗(Λstd) ≤2.AcknowledgmentI thank A. Papageorgiou, S. Paskov, L. Plaskota, V. N. Temlyakov, J. F. Traub,G.

W. Wasilkowski, and A. G. Werschulz for valuable comments.References1. K. I. Babenko, On the approximation of a class of periodic functions of several variablesby trigonometric polynomials, Dokl.

Akad. Nauk SSSR 132 (1960), 247–250, 982–985;English transl.

in Soviet Math. Dokl.

1 (1960).2. D. Lee, Approximation of linear operators on a Wiener space, Rocky Mountain J. Math.16 (1986), 641–659.3.

A. Papageorgiou, Average case complexity bounds for continuous problems, Ph.D. thesis,Dept. of Computer Science, Columbia University, 1990.4.

A. Papageorgiou and G. W. Wasilkowski, On the average complexity of multivariate prob-lems, J. Complexity 6 (1990), 1–23.5.

S. Paskov, Average case complexity of multivariate integration for smooth functions, (toappear in J. Complexity, 1993).6.

P. Speckman, Lp approximation of autoregressive Gaussian processes, Ph.D. thesis, Dept.of Math., UCLA, 1976.7. V. N. Temlyakov, Approximate recovery of periodic functions of several variables, Math.USSR-Sb.

56 (1987), 249–261.8., Private communication, 1991.9. J. F. Traub, G. W. Wasilkowski, and H. Wo´zniakowski, Information-based complexity,Academic Press, New York, 1988.10.

G. Wahba, Interpolating surfaces : high order convergence rates and their associated de-signs, with application to X-ray image reconstruction, Dept. of Statistics, University ofWisconsin, 1978.11., Spline models for observational data, CBMS-NSF Regional Conf.

Ser. in Appl.Math., vol.

59, SIAM, Philadelphia, PA, 1990.12. G. W. Wasilkowski, Randomization for continuous problems, J.

Complexity 5 (1989),195–218.13., Integration and approximation of multivariate functions : average case complexitywith isotropic Wiener measure, Bull. Amer.

Math. Soc.

(N.S.) 28 (1993) (to appear).14.

A. G. Werschulz, Counterexamples in optimal quadratures, Aequationes Math. 29 (1985),183–202.15.

H. Wo´zniakowski, Average case complexity of multivariate integration, Bull. Amer.

Math.Soc. (N.S.) 24 (1991), 185–194.

8H. WO´ZNIAKOWSKI16., Average case complexity of linear multivariate problems, Part I: Theory, Part II:Applications, Dept.

of Computer Science, Columbia University, J. Complexity 8 (1992),337–392.Department of Computer Science, Columbia University and Institute of AppliedMathematics, University of WarsawE-mail address: henryk@ground.cs.columbia.edu

---

출처: arXiv:9307.234 • 원문 보기