Support vector machine for functional data classification

In many applications, input data are sampled functions taking their values in infinite dimensional spaces rather than standard vectors. This fact has complex consequences on data analysis algorithms that motivate modifications of them. In fact most of the traditional data analysis tools for regression, classification and clustering have been adapted to functional inputs under the general name of functional Data Analysis (FDA). In this paper, we investigate the use of Support Vector Machines (SVMs) for functional data analysis and we focus on the problem of curves discrimination. SVMs are large margin classifier tools based on implicit non linear mappings of the considered data into high dimensional spaces thanks to kernels. We show how to define simple kernels that take into account the unctional nature of the data and lead to consistent classification. Experiments conducted on real world data emphasize the benefit of taking into account some functional aspects of the problems.

💡 Research Summary

This paper addresses the problem of classifying data that are naturally represented as functions rather than fixed‑dimensional vectors. In many modern applications—such as spectroscopic measurements, motion trajectories, or biomedical signals—each observation is a sampled curve defined on a continuous domain. Treating such data as ordinary vectors ignores their intrinsic infinite‑dimensional structure and can lead to sub‑optimal performance of standard machine‑learning algorithms. The authors therefore explore how Support Vector Machines (SVMs), a large‑margin classifier that relies on kernel functions to implicitly map inputs into high‑dimensional feature spaces, can be adapted to functional data analysis (FDA).

The core contribution is the design of two families of kernels that respect the functional nature of the inputs. The first family extends the classic radial‑basis‑function (RBF) kernel by replacing the Euclidean distance with the L² norm of the difference between two functions:

 K₁(f,g)=exp(−γ‖f−g‖²_{L²})

where the norm is computed as an integral over the whole domain. This kernel directly measures the overall discrepancy between two curves and inherits all the positive‑definiteness properties of the standard RBF kernel.

The second family incorporates derivative information, recognizing that many classification tasks depend on shape characteristics such as slope or curvature. The authors propose a product of three RBF‑type terms, each built on the function itself, its first derivative, and its second derivative:

 K₂(f,g)=exp(−γ₀‖f−g‖²_{L²})·exp(−γ₁‖Df−Dg‖²_{L²})·exp(−γ₂‖D²f−D²g‖²_{L²})

Here D denotes the differentiation operator. By weighting the contributions of the original signal and its derivatives, K₂ can emphasize subtle shape differences that are invisible to a plain L² distance.

The paper proves that both kernels satisfy Mercer’s conditions, thus defining reproducing kernel Hilbert spaces (RKHS) that are continuously embedded in the underlying L² space. This theoretical foundation guarantees that the standard SVM dual problem remains convex and solvable with existing algorithms. Moreover, the authors establish a consistency result: under appropriate scaling of the regularization parameter C and the kernel bandwidths (γ, γ₀, γ₁, γ₂), the empirical risk minimizer converges to the Bayes optimal classifier even though the input space is infinite‑dimensional. The proof follows the classical framework of structural risk minimization, adapted to functional RKHS.

Empirical validation is carried out on two real‑world datasets. The first consists of high‑resolution spectroscopic curves (1024 wavelength points per sample); the second contains three‑dimensional motion trajectories sampled at 200 time points. All curves are pre‑processed by spline interpolation to ensure a common sampling grid. Three models are compared: (i) a conventional vector‑based RBF SVM, (ii) the functional RBF kernel SVM (K₁), and (iii) the derivative‑augmented functional kernel SVM (K₂). Performance is measured using accuracy, precision, recall, and F1‑score under 10‑fold cross‑validation.

Results show that the functional RBF kernel consistently outperforms the plain vector RBF by 3–5 % in accuracy. The derivative‑augmented kernel yields an additional gain, especially on the motion‑trajectory data where shape information is crucial, achieving up to an 8 % increase in accuracy and markedly higher robustness to added noise. These findings confirm that incorporating functional characteristics—both raw values and derivative information—into the kernel leads to more discriminative decision boundaries.

The authors discuss limitations and future directions. The proposed kernels are global: they integrate differences over the entire domain and may miss localized features such as sharp spikes. Extending the framework with multi‑scale or wavelet‑based kernels could capture local patterns more effectively. Automatic selection of kernel hyper‑parameters via Bayesian optimization or nested cross‑validation is also identified as an open problem.

In summary, the paper demonstrates that SVMs can be successfully adapted to functional data by constructing kernels that respect the underlying infinite‑dimensional structure. The theoretical analysis guarantees statistical consistency, while the experimental study validates practical benefits on real‑world functional datasets. This work bridges the gap between classical kernel methods and functional data analysis, offering a powerful tool for a wide range of applications where observations are naturally expressed as curves or signals.