Functional learning through kernels

This paper reviews the functional aspects of statistical learning theory. The main point under consideration is the nature of the hypothesis set when no prior information is available but data. Within this framework we first discuss about the hypothesis set: it is a vectorial space, it is a set of pointwise defined functions, and the evaluation functional on this set is a continuous mapping. Based on these principles an original theory is developed generalizing the notion of reproduction kernel Hilbert space to non hilbertian sets. Then it is shown that the hypothesis set of any learning machine has to be a generalized reproducing set. Therefore, thanks to a general “representer theorem”, the solution of the learning problem is still a linear combination of a kernel. Furthermore, a way to design these kernels is given. To illustrate this framework some examples of such reproducing sets and kernels are given.

💡 Research Summary

The paper revisits the foundations of statistical learning theory under the austere assumption that no prior knowledge is available beyond the observed data. In this setting the authors focus on the nature of the hypothesis set that a learning machine may employ. They argue that the hypothesis set should be regarded as a vector space of point‑wise defined functions equipped with a continuous evaluation functional, i.e., the map that sends a function f to its value f(x) at any input x. This viewpoint does not require the hypothesis set to be a Hilbert space; it merely demands that the evaluation maps be continuous linear functionals, a condition that holds in any Banach space as well.

Building on this observation, the authors develop a theory that generalizes the classical notion of a Reproducing Kernel Hilbert Space (RKHS) to what they call a “generalized reproducing set.” The key technical contribution is a generalized representer theorem. By exploiting the dual‑space representation of continuous linear functionals (instead of the Riesz representation specific to Hilbert spaces), they prove that for any regularized risk functional of the form
\