Cross-entropy optimisation of importance sampling parameters for statistical model checking

Statistical model checking avoids the exponential growth of states associated with probabilistic model checking by estimating properties from multiple executions of a system and by giving results within confidence bounds. Rare properties are often very important but pose a particular challenge for simulation-based approaches, hence a key objective under these circumstances is to reduce the number and length of simulations necessary to produce a given level of confidence. Importance sampling is a well-established technique that achieves this, however to maintain the advantages of statistical model checking it is necessary to find good importance sampling distributions without considering the entire state space. Motivated by the above, we present a simple algorithm that uses the notion of cross-entropy to find the optimal parameters for an importance sampling distribution. In contrast to previous work, our algorithm uses a low dimensional vector of parameters to define this distribution and thus avoids the often intractable explicit representation of a transition matrix. We show that our parametrisation leads to a unique optimum and can produce many orders of magnitude improvement in simulation efficiency. We demonstrate the efficacy of our methodology by applying it to models from reliability engineering and biochemistry.

💡 Research Summary

Statistical model checking (SMC) offers a scalable alternative to exhaustive probabilistic model checking by estimating the satisfaction probability of temporal properties through repeated simulation runs. However, when the property of interest is a rare event—i.e., its probability lies in the range of 10⁻⁶ or lower—the number of required simulations grows dramatically, making the approach impractical. Importance sampling (IS) mitigates this issue by biasing the simulation towards the rare event and correcting the bias with likelihood ratios, but the effectiveness of IS hinges on the choice of the proposal distribution. Existing methods either require an explicit representation of the full transition matrix (which is infeasible for large models) or rely on high‑dimensional policy learning techniques that suffer from slow convergence and high memory consumption.

The paper introduces a novel algorithm that leverages the cross‑entropy (CE) method to automatically tune a low‑dimensional parameter vector defining the importance sampling distribution. The key idea is to parameterise the transition probabilities of the underlying Markov model using a log‑linear form:

 q_θ(s → s′) = exp(θ·f(s, s′)) / Z(s, θ),

where f(s, s′) is a vector of user‑specified features and θ ∈ ℝ^d (typically d = 5–15) is the parameter vector to be optimised. The CE objective is the Kullback‑Leibler divergence between the optimal (but unknown) zero‑variance distribution P* and the current proposal q_θ, which reduces to minimising the expected log‑likelihood under P*:

 L(θ) = –E_{P*}