Stochastic adaptation of importance sampler

Improving efficiency of importance sampler is at the center of research in Monte Carlo methods. While adaptive approach is usually difficult within the Markov Chain Monte Carlo framework, the counterpart in importance sampling can be justified and validated easily. We propose an iterative adaptation method for learning the proposal distribution of an importance sampler based on stochastic approximation. The stochastic approximation method can recruit general iterative optimization techniques like the minorization-maximization algorithm. The effectiveness of the approach in optimizing the Kullback divergence between the proposal distribution and the target is demonstrated using several simple examples.

💡 Research Summary

The paper addresses a central challenge in Monte Carlo integration: how to choose an efficient proposal distribution for importance sampling (IS). Unlike Markov‑chain Monte Carlo, IS does not require a reversible transition kernel, so the proposal can be altered freely after each batch of samples. The authors exploit this flexibility by formulating an iterative adaptation scheme grounded in stochastic approximation (SA). At each iteration t they draw a set of independent particles {x_i} from the current proposal q_{θ_t}(·), compute importance weights w_i = π(x_i)/q_{θ_t}(x_i) with respect to the target density π, and form an unbiased Monte Carlo estimate of the gradient of the Kullback‑Leibler (KL) divergence D_{KL}(π‖q_{θ}) = ∫π(x) log