The Stochastic Reach-Avoid Problem and Set Characterization for Diffusions

In this article we approach a class of stochastic reachability problems with state constraints from an optimal control perspective. Preceding approaches to solving these reachability problems are either confined to the deterministic setting or address almost-sure stochastic requirements. In contrast, we propose a methodology to tackle problems with less stringent requirements than almost sure. To this end, we first establish a connection between two distinct stochastic reach-avoid problems and three classes of stochastic optimal control problems involving discontinuous payoff functions. Subsequently, we focus on solutions of one of the classes of stochastic optimal control problems—the exit-time problem, which solves both the two reach-avoid problems mentioned above. We then derive a weak version of a dynamic programming principle (DPP) for the corresponding value function; in this direction our contribution compared to the existing literature is to develop techniques that admit discontinuous payoff functions. Moreover, based on our DPP, we provide an alternative characterization of the value function as a solution of a partial differential equation in the sense of discontinuous viscosity solutions, along with boundary conditions both in Dirichlet and viscosity senses. Theoretical justifications are also discussed to pave the way for deployment of off-the-shelf PDE solvers for numerical computations. Finally, we validate the performance of the proposed framework on the stochastic Zermelo navigation problem.

💡 Research Summary

The paper tackles a class of stochastic reach‑avoid problems in which a controlled diffusion must reach a target set while respecting state‑space constraints, but the probabilistic requirements are weaker than the almost‑sure guarantees commonly assumed in the literature. The authors first formalize two distinct reach‑avoid formulations: (i) a “probabilistic reach‑avoid” where the probability of reaching the target without entering a forbidden region must exceed a prescribed threshold, and (ii) an “absolute avoidance” where any entry into the forbidden region before reaching the target is prohibited. They then show that each formulation can be recast as an optimal control problem with a discontinuous payoff, and that all three resulting control problems (two based on penalizing target/obstacle violations and one based on an exit‑time cost) are mathematically equivalent.

The central analytical contribution is the development of a weak dynamic programming principle (DPP) that remains valid despite the payoff’s discontinuities. By constructing upper and lower “viscosity” envelopes of the value function, the authors prove a comparison principle that forces the upper and lower envelopes to coincide, thereby establishing that the value function is a unique discontinuous viscosity solution of a Hamilton‑Jacobi‑Bellman (HJB) partial differential equation. The PDE inherits the usual HJB structure but requires a hybrid boundary treatment: Dirichlet conditions prescribe the payoff when the process hits the target or obstacle, while viscosity‑type conditions capture the behavior at the boundary in a way that is compatible with the discontinuous nature of the value function.

Having obtained this PDE characterization, the paper discusses how existing off‑the‑shelf PDE solvers can be employed without modification, because the weak DPP guarantees the stability of numerical schemes even when the solution is discontinuous. The authors illustrate the entire pipeline on the stochastic Zermelo navigation problem, where a vessel must steer to a goal in a random wind field while avoiding hazardous zones. Numerical results demonstrate that the proposed framework achieves target‑reach probabilities well above the prescribed threshold while keeping the probability of obstacle entry essentially zero. Compared with traditional stochastic optimal‑control approaches that either ignore state constraints or enforce almost‑sure safety, the new method yields higher performance and more realistic safety guarantees.

In summary, the paper makes three key contributions: (1) it bridges stochastic reach‑avoid problems with discontinuous‑payoff optimal control, (2) it establishes a weak DPP and a corresponding theory of discontinuous viscosity solutions for the associated HJB equation, and (3) it provides a practical computational pathway that leverages existing PDE solvers for real‑world stochastic navigation and safety‑critical applications. These results open the door to robust, probabilistically‑tuned control strategies in robotics, autonomous vehicles, finance, and any domain where stochastic dynamics and safety constraints intersect.