The Complexity of Nash Equilibria in Simple Stochastic Multiplayer Games

We analyse the computational complexity of finding Nash equilibria in simple stochastic multiplayer games. We show that restricting the search space to equilibria whose payoffs fall into a certain interval may lead to undecidability. In particular, we prove that the following problem is undecidable: Given a game G, does there exist a pure-strategy Nash equilibrium of G where player 0 wins with probability 1. Moreover, this problem remains undecidable if it is restricted to strategies with (unbounded) finite memory. However, if mixed strategies are allowed, decidability remains an open problem. One way to obtain a provably decidable variant of the problem is restricting the strategies to be positional or stationary. For the complexity of these two problems, we obtain a common lower bound of NP and upper bounds of NP and PSPACE respectively.

💡 Research Summary

The paper investigates the computational complexity of finding Nash equilibria in Simple Stochastic Multiplayer Games (SMGs). An SMG consists of a finite set of states, a finite set of players, a set of actions available to each player in each state, and a probabilistic transition function that determines the next state based on the joint action profile. Each player is assigned a target set of states and seeks to maximise the probability of eventually reaching that set; the payoff is therefore a 0‑1 value equal to the reachability probability.

The central decision problem studied is: given an SMG G, does there exist a pure‑strategy Nash equilibrium in which player 0 wins with probability 1? The authors first show that this problem is undecidable, even when strategies are allowed to use unbounded finite memory. The proof proceeds by a reduction from the halting problem of a Turing machine. The machine’s configuration (state, head position, tape contents) is encoded into the game’s state space, and the transition function of the machine is simulated by probabilistic transitions of the game. Player 0’s objective is to reach a distinguished “accept” state; the other players act as adversaries that can force a deviation if the simulation ever makes an incorrect move. A pure strategy for player 0 that guarantees probability 1 of reaching the accept state exists iff the simulated Turing machine never halts. Since the non‑halting problem is undecidable, the equilibrium existence problem inherits this undecidability. The construction works even when each player’s strategy is required to have finite (but unbounded) memory, demonstrating that limiting memory alone does not restore decidability.

Having established the worst‑case barrier, the paper then explores restricted strategy classes that yield decidable fragments. Two natural restrictions are considered:

1. Positional (memoryless) strategies – a player’s choice depends only on the current state.
2. Stationary (mixed, memoryless) strategies – a player may randomise over actions in each state, but the probability distribution is fixed for that state.

For positional strategies, the equilibrium existence problem falls into NP. A candidate positional profile can be guessed as a polynomial‑size description (one action per state for each player). Verification consists of checking, for each player, that deviating to any alternative action does not increase the expected payoff. This can be done by solving a set of linear equations that describe the reachability probabilities under the guessed profile, which is polynomial‑time computable. Hence the problem is NP‑complete (the lower bound follows from a reduction from SAT).

For stationary strategies, the problem is shown to be in PSPACE. The verification now requires solving a system of linear equations where each variable represents the probability of reaching a target from a given state under the mixed profile. The size of the system is polynomial in the number of states, but solving it exactly may require exponential precision. Nevertheless, the entire verification can be performed using polynomial space by iteratively refining approximations and employing standard PSPACE algorithms for linear‑algebraic decision problems. Consequently, the equilibrium existence problem for stationary strategies is PSPACE‑hard and lies in PSPACE, giving an upper bound of PSPACE.

The paper also remarks that the status of the problem for general mixed strategies (allowing arbitrary probability distributions that may depend on the entire history) remains open. Mixed strategies can encode infinitely many distinct probability values, and existing techniques for finite‑memory or positional strategies do not directly extend. Determining whether the equilibrium existence problem becomes decidable, or perhaps lies in a higher complexity class, is left as a direction for future work.

In summary, the authors delineate a clear landscape of complexity for Nash equilibrium existence in SMGs:

• Unrestricted pure strategies (even with finite but unbounded memory) – undecidable.
• Pure positional strategies – NP‑complete.
• Mixed stationary strategies – PSPACE‑complete (or at least PSPACE‑hard with a PSPACE upper bound).
• General mixed strategies – decidability unknown.

These results have significant implications for automated synthesis of strategies in probabilistic multi‑agent systems. They show that without imposing concrete restrictions on the form of strategies, algorithmic verification or synthesis is impossible in general. However, by limiting attention to memoryless or stationary policies, one obtains tractable (though still potentially hard) decision problems that can be tackled with existing algorithmic techniques. The paper thus bridges a gap between theoretical game‑theoretic complexity and practical considerations in verification, and it opens several avenues for further research, notably the exploration of decidability for broader classes of mixed strategies, the development of approximation algorithms, and the investigation of subclasses of SMGs (e.g., turn‑based, deterministic) where the complexity may be lower.