Multiple Kernel Learning: A Unifying Probabilistic Viewpoint

We present a probabilistic viewpoint to multiple kernel learning unifying well-known regularised risk approaches and recent advances in approximate Bayesian inference relaxations. The framework proposes a general objective function suitable for regression, robust regression and classification that is lower bound of the marginal likelihood and contains many regularised risk approaches as special cases. Furthermore, we derive an efficient and provably convergent optimisation algorithm.

💡 Research Summary

This paper presents a unified probabilistic framework that bridges the two dominant paradigms in kernel learning: regularised risk minimisation (the “classical” SVM/ridge‑regression view) and Bayesian Gaussian‑process (GP) modelling. The authors start by formalising the kernel as a linear combination of base kernels, K(θ)=∑ₘθₘKₘ, with non‑negative coefficients θ. In the regularised‑risk setting the objective is
  uᵀK(θ)^{-1}u + C∑₁ⁿℓ(yᵢ,uᵢ) ,
where ℓ is a loss function and C a trade‑off parameter. By the representer theorem the solution can be expressed as u=K(θ)α, turning the problem into a finite‑dimensional optimisation over α.

From a Bayesian perspective the same setting is described by a GP prior P(u|θ)=𝒩(u|0,K(θ)) and a likelihood P(y|u). The joint MAP estimate then minimises
  uᵀK(θ)^{-1}u – 2∑₁ⁿ log P(yᵢ|uᵢ) + log|K(θ)| .
The log‑determinant term log|K(θ)| is the normalising constant of the prior; it grows without bound when any θₘ→∞, thereby automatically penalising overly complex kernels (an Occam’s‑razor effect).

Multiple kernel learning (MKL) traditionally adds a convex regulariser λ‖θ‖ₚᵖ (or an ℓₚ‑norm ball constraint) to the risk objective, yielding
  φ_MKL(θ)=min_u