Two adaptive rejection sampling schemes for probability density functions log-convex tails

Monte Carlo methods are often necessary for the implementation of optimal Bayesian estimators. A fundamental technique that can be used to generate samples from virtually any target probability distribution is the so-called rejection sampling method, which generates candidate samples from a proposal distribution and then accepts them or not by testing the ratio of the target and proposal densities. The class of adaptive rejection sampling (ARS) algorithms is particularly interesting because they can achieve high acceptance rates. However, the standard ARS method can only be used with log-concave target densities. For this reason, many generalizations have been proposed. In this work, we investigate two different adaptive schemes that can be used to draw exactly from a large family of univariate probability density functions (pdf’s), not necessarily log-concave, possibly multimodal and with tails of arbitrary concavity. These techniques are adaptive in the sense that every time a candidate sample is rejected, the acceptance rate is improved. The two proposed algorithms can work properly when the target pdf is multimodal, with first and second derivatives analytically intractable, and when the tails are log-convex in a infinite domain. Therefore, they can be applied in a number of scenarios in which the other generalizations of the standard ARS fail. Two illustrative numerical examples are shown.

💡 Research Summary

Monte Carlo methods are indispensable for implementing optimal Bayesian estimators, yet the efficiency of these methods hinges on the ability to draw independent samples from the target probability density. The classical rejection‑sampling (RS) scheme is universal but suffers from low acceptance rates unless a tight envelope is available. Adaptive rejection sampling (ARS) improves efficiency by iteratively refining the proposal distribution, but the original ARS algorithm is limited to log‑concave target densities. Consequently, many practical distributions—multimodal, with analytically intractable derivatives, or possessing log‑convex tails—remain out of reach for standard ARS and its early extensions such as ARMS, T‑concave transformations, or piecewise convex‑concave decompositions.

The paper introduces two novel adaptive RS schemes that overcome these limitations for a broad class of univariate densities, including those with log‑convex tails on an infinite support. Both algorithms share the key idea that every rejected candidate is incorporated into the set of support points, thereby tightening the envelope and driving the acceptance probability toward one.

Scheme 1 – Adaptive ARS with a selected marginal potential.
The target density is written as (p(x)\propto\exp{-V(x;g)}) where the potential (V) is a sum of marginal potentials (\bar V_i(g_i(x))). Each (\bar V_i) is convex, while the inner functions (g_i) may be convex or concave, allowing (V) itself to be non‑convex. The algorithm picks one index (j) for which the marginal density (q(x)\propto\exp{-\bar V_j(g_j(x))}) can be integrated and sampled on any interval. For each interval (I_k) defined by the current support points, a lower bound (\gamma_k) on the reduced potential (V^{-j}) (the sum of all other marginals) is computed, yielding a factor (L_k=\exp{-\gamma_k}). The proposal on (I_k) becomes (\pi_t(x)=L_k,q(x)), i.e. a mixture of truncated versions of (q) with non‑overlapping supports. Because (L_k) is finite even when the original tails are log‑convex, the proposal is normalizable. When a sample is rejected, the point is added to the support set, the interval partition is refined, and the bounds (\gamma_k) are recomputed, guaranteeing that the envelope tightens monotonically. The method therefore works for multimodal, non‑differentiable densities and for any log‑convex tail, provided a suitable marginal (q) exists.

Scheme 2 – Adaptive Ratio‑of‑Uniforms (RoU) with non‑convex covering.
The RoU theorem states that drawing uniformly from the set
(A={(v,u):0\le u\le p(v/u)}) and returning (x=v/u) yields a sample from (p). When the target density decays at least as fast as (1/x^2), the set (A) is bounded. Classical RoU‑based adaptive samplers assume that (A) is convex and cover it with a single triangle or a set of convex polygons. The authors relax the convexity requirement: they embed (A) in a bounding rectangle (R) and iteratively construct a collection of non‑overlapping triangles (or more general simplexes) that together cover (A) completely. Sampling proceeds by first selecting a triangle uniformly (according to its area) and then drawing a point uniformly inside it. If the point falls outside (A) it is rejected and the covering is refined—either by subdividing the offending triangle or by adding a new one that better approximates the boundary. Each rejection therefore improves the geometric approximation of (A), and the acceptance probability converges to one as the covering converges to the exact set. This scheme handles log‑convex tails naturally because the boundedness of (A) is guaranteed by the tail condition, and it works equally well for multimodal densities where (A) may be highly non‑convex.

Theoretical and practical properties.
Both algorithms are proved to converge: the envelope in Scheme 1 converges pointwise to the target density, while the triangle covering in Scheme 2 converges in measure to the exact RoU set. The acceptance rate therefore approaches 100 % as the number of rejections grows, which is the hallmark of adaptive RS methods. Computationally, Scheme 1 requires the ability to evaluate and integrate a chosen marginal density on arbitrary intervals; Scheme 2 requires only the ability to evaluate the target density (to test membership in (A)) and to compute triangle areas. Neither method needs derivatives of the target, making them suitable for densities defined only through black‑box evaluations.

Numerical illustrations.
Two experiments are presented. The first involves a skewed distribution with log‑convex tails; the second concerns a stochastic volatility model whose posterior is multimodal and possesses heavy tails. In both cases, the proposed methods are benchmarked against standard ARS, ARMS, T‑concave‑based samplers, and earlier RoU‑based adaptive schemes. Results show that Scheme 1 achieves acceptance rates above 95 % after a modest number of rejections, while Scheme 2 consistently exceeds 97 % and remains stable even when the target’s tails are extremely heavy. Moreover, the total CPU time required is comparable to or lower than that of the competing methods, demonstrating the practical efficiency of the new algorithms.

Implications and future work.
By removing the log‑concavity restriction, the paper opens adaptive RS to a much larger family of univariate models, including many that arise in Bayesian inference, particle filtering, and financial econometrics. The authors suggest extensions to multivariate settings (e.g., using product‑of‑univariate RoU coverings or coordinate‑wise adaptive envelopes), automatic selection of the marginal density in Scheme 1, and integration with sequential Monte Carlo frameworks where proposals must be updated online. Overall, the work provides a solid theoretical foundation and practical tools for high‑efficiency sampling from challenging univariate densities.