Stochastic Optimization with Optimal Importance Sampling

Importance Sampling (IS) is a widely used variance reduction technique for enhancing the efficiency of Monte Carlo methods, particularly in rare-event simulation and related applications. Despite its effectiveness, the performance of IS is highly sensitive to the choice of the proposal distribution and often requires stochastic calibration. While the design and analysis of IS have been extensively studied in estimation settings, applying IS within stochastic optimization introduces a lesser-known fundamental challenge: the decision variable and the importance sampling distribution are mutually dependent, creating a circular optimization structure. This interdependence complicates both convergence analysis and variance control. In this paper, we consider the generic setting of convex stochastic optimization with linear constraints. We propose a single-loop stochastic approximation algorithm, based on a variant of Nesterov’s dual averaging, that jointly updates the decision variable and the importance sampling distribution, notably without time-scale separation or nested optimization. The method is globally convergent and achieves the minimal asymptotic variance among stochastic gradient schemes, which moreover matches the performance of an oracle sampler adapted to the optimal solution and thus effectively resolves the circular optimization challenge.

💡 Research Summary

The paper addresses a fundamental difficulty that arises when importance sampling (IS) is incorporated into stochastic optimization: the decision variable and the IS proposal distribution depend on each other, creating a circular optimization structure. Existing approaches either assume a known analytical mapping from the decision to an optimal IS distribution—typically derived from large‑deviation theory—or employ nested or two‑time‑scale algorithms (e.g., cross‑entropy methods combined with stochastic approximation). Both strategies limit applicability and complicate convergence analysis, especially in the presence of linear constraints that may cause abrupt changes in the optimal sampling distribution.

To overcome these limitations, the authors propose a single‑loop stochastic approximation algorithm that jointly updates the decision variable θ and the IS parameters λ. The method is built on a variant of Nesterov’s dual averaging (NDA). At each iteration the algorithm draws a sample X from the current IS distribution Q_λ, evaluates the loss F(θ,X) and its gradient with respect to θ, and computes the gradient of the log‑likelihood ratio ∇_λ log (dP/dQ_λ). Both gradients are fed into a dual‑averaging update; the θ‑update is then projected onto the feasible set Θ = {θ | Aθ ≤ b}. Crucially, the updates for θ and λ are performed simultaneously, eliminating the need for inner loops or separate time‑scale schedules.

The theoretical contributions are threefold. First, under standard assumptions—convexity and twice continuous differentiability of the objective f(θ)=E_P