Mathematics / math.ST Statistics / Machine Learning Statistics / stat.TH

PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers

Reading time: 3 minute
...
#Statistics #Mathematics #Machine Learning

📝 Original Info

  • Title: PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers
  • ArXiv ID: 0712.1698
  • Date: 2007-12-11
  • Authors: Pierre Alquier

📝 Abstract

The aim of this paper is to generalize the PAC-Bayesian theorems proved by Catoni in the classification setting to more general problems of statistical inference. We show how to control the deviations of the risk of randomized estimators. A particular attention is paid to randomized estimators drawn in a small neighborhood of classical estimators, whose study leads to control the risk of the latter. These results allow to bound the risk of very general estimation procedures, as well as to perform model selection.

💡 Deep Analysis

Deep Dive into PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers.

The aim of this paper is to generalize the PAC-Bayesian theorems proved by Catoni in the classification setting to more general problems of statistical inference. We show how to control the deviations of the risk of randomized estimators. A particular attention is paid to randomized estimators drawn in a small neighborhood of classical estimators, whose study leads to control the risk of the latter. These results allow to bound the risk of very general estimation procedures, as well as to perform model selection.

📄 Full Content

Ò Ø ÓÒ ½º½º Ä Ø Ù× ÔÙØ

Ü ÑÔÐ ½º¿ ´ Ò× ØÝ ×Ø Ñ Ø ÓÒµº À Ö ¸Û ××ÙÑ Ø Ø P 1 = … = P N = P Ò ÓÒ× ÕÙ ÒØÐÝ Ø Ø

.

P exp sup

´ º½µ

´ º¾µ

´ º¿µ

´ º µ β -

, θi,β -βr ′ (., θ)

-log π i exp -βr ′ (., θ) .

´ º½¿µ

´ º½ µ R ′ θt , θt j ≤ B(t j , t s(j) ) + B(t s(j) , t j ) + B( t, t ŝ) + B(t ŝ, t) .

Ï Ö Ó Ò ØÓ ÙÔÔ Ö ÓÙÒ × Ô Ö Ø ÐÝ B(t j , t s(j) ) + B(t s(j) , t j ) Ò B( t, t ŝ) + B(t ŝ, t)º Ä Ø Ù× Ö×Ø Ð Û Ø Ø Ø ÖÑ B(t j , t s(j) ) + B(t s(j) , t j )

´ º½ µ B(t j , t s(j) ) + B(t s(j) , t j) , θt j ) ≤ B(t j , t s(j) ) + B(t s(j) , t j ) ≤ Eδ N (i, q, ε, κ).

ÈÐÙ Ò ÐÐ Ø × Ö ×ÙÐØ× ÒØÓ ÁÒ ÕÙ Ð ØÝ ´ º½ µ¸Û Ó Ø Ò¸ 1 -2λx N R ′ θt , θt j ≤ Eδ N (i, q, ε, κ) + 2λ N xEδ N (i, q, ε, κ) + 2xR ′ ( θt j , θ i ) + 2xR ′ (θ i , θ) + ϕ(x)

.

× Ù×Ù Ð¸Ð Ø Ù× ÔÔÐÝ Ä ÑÑ º ØÓ ÙÔÔ Ö ÓÙÒ C(t j )¸Ä ÑÑ º¿ ØÓ ÙÔÔ Ö ÓÙÒ R ′ ( θt j , θ i ) Ò Ä ÑÑ º½ ØÓ ÙÔÔ Ö ÓÙÒ ϕ(x)º Ä Ø Ù× ÔÙØ

θt , θt j ≤ E ′ δ N (i, q, ε, κ). Ì Ò¸ ÓÖ ÒÝ (θ, θ ′ ) ∈ Θ 2 Ŗ(θ)

📸 Image Gallery

cover.png

Reference

This content is AI-processed based on ArXiv data.

Related Posts

Start searching

Enter keywords to search articles

↑↓
↵
ESC
⌘K Shortcut