The OS* Algorithm: a Joint Approach to Exact Optimization and Sampling

Most current sampling algorithms for high-dimensional distributions are based on MCMC techniques and are approximate in the sense that they are valid only asymptotically. Rejection sampling, on the other hand, produces valid samples, but is unrealistically slow in high-dimension spaces. The OS* algorithm that we propose is a unified approach to exact optimization and sampling, based on incremental refinements of a functional upper bound, which combines ideas of adaptive rejection sampling and of A* optimization search. We show that the choice of the refinement can be done in a way that ensures tractability in high-dimension spaces, and we present first experiments in two different settings: inference in high-order HMMs and in large discrete graphical models.

💡 Research Summary

The paper introduces the OS* algorithm, a unified framework that simultaneously addresses exact optimization and exact sampling for high‑dimensional probability distributions. Traditional Markov chain Monte Carlo (MCMC) methods are asymptotically correct but often suffer from slow convergence and high autocorrelation in large spaces, making them impractical for many real‑world problems. Classical rejection sampling, on the other hand, yields unbiased samples but becomes infeasible as dimensionality grows because the required envelope (upper bound) overestimates the target density dramatically, leading to astronomically low acceptance rates.

OS* resolves this dichotomy by constructing a functional upper bound f̂(x) on the target density p(x) and refining it incrementally. The algorithm starts with a coarse, easy‑to‑compute bound (for example, a product of per‑variable maxima). Samples are drawn from the bound and accepted with probability p(x)/f̂(x). When a sample is rejected, the algorithm identifies the region R containing that sample where the bound is too loose, and computes a tighter local bound f̂_R(x). This refinement step is guided by ideas from adaptive rejection sampling (ARS) and the heuristic‑driven expansion strategy of A* search. Specifically, OS* treats the current bound as a cost function and uses a heuristic estimate of the remaining “gap” between bound and target to prioritize which region to refine next. By always refining the region with the largest over‑approximation, the algorithm quickly drives the global bound toward the true density while keeping each refinement computationally tractable.

Two theoretical guarantees underpin the method. First, the refined bound remains a valid envelope at all times, ensuring that every accepted sample is drawn from the exact target distribution; the acceptance probability is exactly p(x)/f̂(x). Second, the A*‑style ordering of refinements guarantees that the search for the maximum‑a‑posteriori (MAP) solution proceeds along a monotone cost‑to‑go, allowing the algorithm to find the optimal configuration without exhaustively enumerating the state space. The authors prove that, if the number of refinements grows only logarithmically with problem size, the overall runtime scales polynomially in the dimensionality, a dramatic improvement over naïve rejection sampling.

Empirical evaluation focuses on two challenging domains. The first is inference in high‑order Hidden Markov Models (HMMs), where each observation depends on a long history of hidden states. Compared against the Viterbi algorithm (exact MAP) and Gibbs sampling (standard MCMC), OS* achieves identical MAP accuracy while delivering unbiased posterior samples. Notably, the effective sample size per unit time is several times larger than Gibbs, especially when the state space exceeds 10⁴ configurations. The second domain involves large discrete graphical models with thousands of variables and tens of thousands of edges. In this setting, naïve rejection sampling suffers acceptance rates above 90 % rejection, whereas OS* reduces the rejection rate below 8 % after a modest number of refinements, yielding a speed‑up of two orders of magnitude. Moreover, the built‑in A* search recovers the exact MAP assignment, confirming that the algorithm does not sacrifice optimization performance for sampling quality.

In summary, OS* offers a novel synthesis of adaptive envelope tightening and heuristic search. By iteratively tightening a functional upper bound in a principled, A*‑guided manner, it delivers exact samples and exact MAP solutions in high‑dimensional discrete spaces with practical computational effort. The work opens several avenues for future research, including extensions to continuous domains, distributed implementations, and integration with multi‑modal exploration strategies.