Approximating Data with weighted smoothing Splines

In this paper we show how the goal of local adaptivity of the function and its first and second derivatives can be attained in a simple manner using weighted smoothing splines. A residual based concept of approximation is used which forces local adaptivity of the regression function together with a global regularization which makes the function as smooth as possible subject to the approximation constraints.

AMS 2000 Subject classifications: Primary 62G08, secondary 62G15, 62G20

1 Introduction

In the one-dimensional case nonparametric regression is concerned with determining a function f n : [0, 1] → R which adequately represents a data set y n = {(t i , y(t i )) :

The problem is to provide a function f n which is an adequate representation of the data. One well established method for accomplishing this goal is that of smoothing splines defined as the solution of the problem min S(g, λ) :

where λ is the smoothing parameter (see Wahba (1990); Green and Silverman (1994); Ruppert et al. (2003)). This approach has two weaknesses. The first is that there may not be any choice of λ for which the resulting fit is satisfactory. This is particularly the case if the data show large local variations such as in Figure 1 which are taken from thin film physics. They were kindly supplied by Prof. Dieter Mergel of the Department of Physics, University of Duisburg-Essen. X-rays are beamed onto a thin film and the data give the photon count of the diffracted rays as a function of the angle of diffraction. The sample size is n = 7001. The high peaks can only be adequately captured with a small value of λ in (1). This has however the consequence that the function oscillates too rapidly between the peaks. The second problem is to give an automatic choice for λ. Methods suggested include cross-validation, generalized cross-validation, generalized maximum likelihood and restricted maximum likelihood (Craven and Wahba (1978); Wahba (1985); Ruppert et al. (2003)). However it is clear that if there is no satisfactory value of λ then no automatic choice will work. In this paper we attain more flexibility by considering a vector λ = (λ 1 , . . . , λ n ) rather than a single value λ and we replace the minimization problem (1) by min S(g, λ) := n i=1 λ i (y(t i ) -g(t i )) 2 + 1 0 g (2) (t) 2 dt.

(2)

Comparing this with (1) we see that the smoothing parameter λ has now been transferred from the penalty term to the observations themselves. The solution, which we denote by f n (• : λ), is a natural cubic spline (see Green and Silverman (1994)) but the λ i now control the fit at the observation points (t i , y(t i )) rather than the size of the penalty which is now fixed. In the case of the data displayed in Figure 1 we would choose large values of λ i at the peaks causing them to be adequately approximated. At points away from the peaks we would choose the λ i to be small and thus ensure a smooth solution at these points.

The method proposed here belongs to the category of spatially adaptive splines.

For other spatially adaptive spline methods we refer to Luo and Wahba (1997), Denison et al. (1998), Ruppert and Carroll (2000), Zhou andShen (2001), DiMatteo et al. (2001), Pittman (2002), Wood et al. (2002), Miyata andShen (2003, 2005), Pintore et al. (2006).

In Section 2 we describe an approach to choosing a model in the context of nonparametric regression which is based on a universal, honest and non-asymptotic confidence region. Section 3 shows how the ideas of Section 2 can be adapted to give a simple method for choosing the weights of a weighted smoothing spline. Examples and the results of a small simulation study are given in Section 4. Section 5 gives two variations on this theme and Section 6 extends the method to image analysis. Finally in Section 7 we look at the asymptotics.

A lot of work has been devoted to choosing a model from a sequence of models of increasing complexity. Choosing a value of λ in (1) falls into this category as the smaller λ the more complex the resulting smoothing spline. Methods developed to solve the problem include cross-validation, plug-in methods as well as AIC and BIC which are explicitly phrased in terms of balancing complexity and fidelity. We take a different approach here which is implicit in Davies and Kovac (2004) and explicit in Davies et al. (2008b). We define a universal, honest and non-asymptotic confidence region A n and given thi

…(Full text truncated)…