Computer Science and Game Theory 24 JAN, 2010

Decision Problems for Nash Equilibria in Stochastic Games

Decision Problems for Nash Equilibria in Stochastic Games

We analyse the computational complexity of finding Nash equilibria in stochastic multiplayer games with $\omega$-regular objectives. While the existence of an equilibrium whose payoff falls into a certain interval may be undecidable, we single out several decidable restrictions of the problem. First, restricting the search space to stationary, or pure stationary, equilibria results in problems that are typically contained in PSPACE and NP, respectively. Second, we show that the existence of an equilibrium with a binary payoff (i.e. an equilibrium where each player either wins or loses with probability 1) is decidable. We also establish that the existence of a Nash equilibrium with a certain binary payoff entails the existence of an equilibrium with the same payoff in pure, finite-state strategies.

💡 Research Summary

The paper investigates the algorithmic difficulty of locating Nash equilibria in stochastic multiplayer games whose objectives are expressed by ω‑regular conditions such as Büchi or parity. After formalising the game model—a finite directed graph with a set of players, each equipped with a set of actions, a probabilistic transition function, and an ω‑regular winning condition—the authors focus on the decision problem: given a payoff interval (or a specific payoff vector), does there exist a Nash equilibrium whose expected payoffs fall inside that interval?

In the unrestricted setting, where strategies may use unbounded memory and randomisation, the authors show that this decision problem can be undecidable. This aligns with known results for infinite‑horizon stochastic games: the space of all strategies is so rich that the existence of an equilibrium cannot be captured by any algorithmic procedure.

To obtain tractable fragments, the paper introduces three natural restrictions on the strategy space.

  1. Stationary strategies – strategies that depend only on the current state and prescribe a probability distribution over actions. Under this restriction the equilibrium existence problem can be expressed as a system of linear inequalities over the expected values of each player. The authors prove that checking the feasibility of this system lies in PSPACE. The proof proceeds by constructing a polynomial‑size representation of the system and applying a PSPACE‑bounded fixed‑point computation to verify whether a solution exists.

  2. Pure stationary strategies – a further restriction that eliminates randomisation, forcing each state to be associated with a single deterministic action for each player. The search space becomes finite (though exponential in the number of states). The authors reduce the equilibrium existence problem to a Boolean satisfiability instance and demonstrate that it belongs to NP. Moreover, they establish NP‑hardness by a reduction from 3‑SAT, thereby proving NP‑completeness for this fragment.

  3. Binary‑payoff equilibria – equilibria in which every player either wins with probability 1 or loses with probability 0. This special case is motivated by verification scenarios where a system must guarantee a safety property with certainty. The main technical contribution here is a constructive proof that if a binary‑payoff equilibrium exists at all (even with arbitrary strategies), then there also exists one that can be realised by pure strategies using only a finite amount of memory. The construction proceeds by identifying the set of states from which each player can force his ω‑regular objective with probability 1 (a classic almost‑sure winning set in Markov decision processes). The intersection of all players’ almost‑sure winning sets yields a region where all players can simultaneously achieve certainty. Inside this region, each player’s best response can be encoded as a deterministic finite‑state controller. The authors show that any unilateral deviation cannot improve a player’s payoff, establishing the Nash property. The decision procedure for the existence of such an equilibrium reduces to computing almost‑sure winning sets, which can be done in polynomial time for many ω‑regular conditions; overall the problem lies in PSPACE (and often lower).

Collectively, these results produce a clear complexity landscape: the general equilibrium existence problem is undecidable, but restricting to stationary strategies brings it into PSPACE, restricting further to pure stationary strategies makes it NP‑complete, and the binary‑payoff case is decidable and even admits finite‑state pure implementations.

The paper situates its contributions within the broader literature. Prior work has largely addressed two‑player zero‑sum stochastic games or single‑objective settings; the present study is the first to systematically explore the multi‑player, non‑zero‑sum case with ω‑regular objectives. By highlighting how the choice of strategy class dramatically influences decidability and complexity, the authors provide practical guidance for designers of automated synthesis tools, model‑checking frameworks, and security protocols that rely on equilibrium reasoning.

Finally, the authors outline future directions, including extending the analysis to bounded‑memory strategies, other quantitative objectives such as mean‑payoff or discounted reward, and investigating approximation algorithms for cases where exact equilibrium computation remains intractable. The work thus opens a rich line of inquiry at the intersection of game theory, formal verification, and algorithmic complexity.