A Method for Estimating the Probability of Extremely Rare Accidents in Complex Systems

Estimating the probability of failures or accidents with aerospace systems is often necessary when new concepts or designs are introduced, as it is being done for Autonomous Aircraft. If the design is safe, as it is supposed to be, accident cases are hard to find. Such analysis needs some variance reduction technique and several algorithms exist for that, however specific model features may cause difficulties in practice, such as the case of system models where independent agents have to autonomously accomplish missions within finite time, and likely with the presence of human agents. For handling these scenarios, this paper presents a novel estimation approach, based on the combination of the well-established variation reduction technique of Interacting Particles System (IPS) with the long-standing optimization algorithm denominated DIviding RECTangles (DIRECT). When combined, these two techniques yield statistically significant results for extremely low probabilities. In addition, this novel approach allows the identification of intermediate events and simplifies the evaluation of sensitivity of the estimated probabilities to certain system parameters.

💡 Research Summary

**
The paper addresses the challenging problem of estimating extremely low probabilities (on the order of 10⁻⁹ or smaller) of accidents in complex socio‑technical systems such as autonomous aircraft. Traditional rare‑event techniques—importance splitting, cross‑entropy, Markov‑chain Monte Carlo, extrapolation—either lack guarantees of finding the global optimum, suffer from particle degeneracy, or require assumptions (e.g., stationary state distributions) that do not hold for finite‑time, human‑in‑the‑loop missions. To overcome these limitations, the authors propose a hybrid framework that couples the Interacting Particle System (IPS), a sequential Monte Carlo variance‑reduction method, with the global optimisation algorithm DIviding RECTangles (DIRECT).

The methodology proceeds in six clearly defined steps: (1) construct a multi‑agent dynamic risk model (MA‑DRM) that captures both technical components and human agents; (2) identify purely stochastic input parameters (those without recursive definitions); (3) assign probability distributions to these parameters based on expert judgement or data; (4) define a scalar objective function that quantifies proximity to failure (e.g., minimum aircraft‑terrain distance, reactor core temperature); (5) use DIRECT to globally search for minima of the objective function while recursively partitioning the parameter space into hyper‑rectangles; (6) within each rectangle, run IPS to generate weighted particles, resample them according to their likelihood of crossing intermediate thresholds, and finally compute the rare‑event probability together with confidence intervals. The partition produced by DIRECT enables rapid recomputation for sensitivity and uncertainty analyses, because the probability contribution of each rectangle can be re‑evaluated without re‑running the full simulation.

A first case study applies the “basic” version of the method (no stochastic differential equations) to a hypothetical transport aircraft flying a U‑shaped route between two mountainous peaks. The aircraft model is a six‑state ODE with a hybrid feedback controller; an altitude sensor fault introduces a deterministic offset, while a constant wind field provides environmental disturbance. The objective is the minimum distance between aircraft and terrain; a collision occurs when this distance reaches zero. DIRECT quickly isolates the region of the input space (wind components, sensor offset) that leads to near‑collision trajectories and refines it to a resolution of 10⁻⁴. By combining the prior probability of each hyper‑rectangle with IPS‑estimated conditional probabilities, the overall collision probability is estimated at ≈1.2 × 10⁻⁹ with a 95 % confidence interval, using only a few thousand model evaluations. By contrast, naïve Monte Carlo would require on the order of 10⁹ runs to achieve comparable statistical accuracy. Sensitivity analysis shows that wind‑x and the altitude‑sensor bias dominate the risk, providing actionable insight for designers.

The authors then extend the approach to systems that include stochastic differential equations (SDEs). In this “advanced” version, DIRECT still performs the global partition, but each rectangle now contains an SDE‑based simulation. IPS is employed to handle intermediate events (e.g., crossing a safety threshold) and to prevent particle collapse by resampling only those particles that have survived the intermediate stages. This hybridisation retains the variance‑reduction benefits of IPS while leveraging DIRECT’s ability to focus computational effort on the most critical regions of the high‑dimensional space. Experiments on an SDE‑augmented scenario demonstrate that probabilities as low as 10⁻⁸ can be estimated with fewer than 10⁴ simulations, again far outperforming conventional techniques.

The paper discusses several practical considerations. The choice of the objective function is crucial; an ill‑chosen metric can misguide the DIRECT search and waste computational resources. The depth of DIRECT’s partition directly influences memory and CPU usage, especially in dimensions higher than 30, suggesting the need for dimensionality‑reduction or adaptive sampling strategies. IPS resampling frequency must be balanced to avoid loss of diversity while still concentrating particles in the rare‑event region. Despite these challenges, the combined DIRECT‑IPS framework offers a systematic, statistically sound, and computationally efficient tool for early‑stage safety assessment of complex designs, enabling designers to verify compliance with Target Levels of Safety (TLS) and to identify the most influential uncertain parameters.

In conclusion, the authors deliver a novel, integrated methodology that bridges global optimisation and sequential Monte Carlo to make the estimation of ultra‑rare event probabilities tractable for multi‑agent, human‑in‑the‑loop systems. The approach delivers orders‑of‑magnitude reductions in required simulations, provides built‑in sensitivity and uncertainty quantification, and is demonstrated on realistic aerospace scenarios. Future work is suggested on automated objective‑function generation, further scalability to very high‑dimensional problems, and real‑time risk monitoring applications.