Making the Best of Limited Memory in Multi-Player Discounted Sum Games

In this paper, we establish the existence of optimal bounded memory strategy profiles in multi-player discounted sum games. We introduce a non-deterministic approach to compute optimal strategy profiles with bounded memory. Our approach can be used to obtain optimal rewards in a setting where a powerful player selects the strategies of all players for Nash and leader equilibria, where in leader equilibria the Nash condition is waived for the strategy of this powerful player. The resulting strategy profiles are optimal for this player among all strategy profiles that respect the given memory bound, and the related decision problem is NP-complete. We also provide simple examples, which show that having more memory will improve the optimal strategy profile, and that sufficient memory to obtain optimal strategy profiles cannot be inferred from the structure of the game.

💡 Research Summary

The paper investigates multi‑player discounted sum games (MDSGs) under realistic resource constraints, namely bounded memory for each player’s strategy. An MDSG is defined on a finite directed graph where each vertex belongs to a specific player; the token moves according to the owner’s chosen action, and every transition yields a vector of rewards that are discounted by a factor λ∈(0,1). The total payoff for a player is the infinite discounted sum of the rewards obtained along the play.

Two equilibrium concepts are considered. A Nash equilibrium requires that no player can improve her expected payoff by unilaterally deviating, assuming all others keep their strategies. A leader equilibrium (also called a leader strategy profile) designates a distinguished “leader” player who may prescribe strategies to all participants, including herself, but the Nash condition is waived for the leader herself. Consequently, every Nash equilibrium is a leader equilibrium, but not vice‑versa; the leader can potentially achieve a higher payoff by exploiting the freedom to change her own strategy after the other players have committed.

The central contributions are:

1. Existence of optimal bounded‑memory leader strategies. The authors prove that for any fixed memory bound M, there exists a leader strategy profile that is optimal among all profiles respecting that bound. The proof relies on a “reward‑and‑punish” construction: the leader first defines a main play (the “reward” phase) that yields a desired payoff, and specifies a punishment sub‑game that is triggered if any non‑leader deviates. The punishment sub‑game is a two‑player zero‑sum discounted game, which is known to admit memoryless optimal strategies. Hence the only memory that needs to be stored concerns the compliance of the players with the prescribed main play.

2. A non‑deterministic polynomial‑time synthesis algorithm. The paper presents a nondeterministic procedure that, given a memory bound M and a payoff threshold τ, guesses a finite memory machine (a Moore‑machine with M states) together with the reward‑and‑punish actions, and then verifies in polynomial time that the resulting leader payoff is at least τ while all other players have no incentive to deviate. This yields an NP algorithm for the decision problem “does there exist a bounded‑memory leader strategy achieving payoff ≥ τ?”.

3. Memory‑payoff trade‑off. Lemma 6 shows that increasing the memory bound can strictly improve the leader’s achievable payoff. The authors illustrate this with concrete examples (Figures 2–4). In one example, a leader with one memory cell can only obtain payoff 0, whereas with two cells she can secure a positive payoff by staying a prescribed number of rounds in a particular vertex before moving on.

4. Necessity of unbounded memory in some games. Theorem 7, 8, and 9 construct families of MDSGs where any optimal Nash or leader equilibrium requires arbitrarily large (in fact, infinite) memory. These constructions are based on cycles whose discounted contribution can be made arbitrarily close to a target value only by counting the number of repetitions, which a bounded counter cannot achieve.

5. Complexity classification. The decision problem “given M and τ, does a bounded‑memory leader strategy exist?” is shown to be NP‑complete. NP‑hardness is proved by reduction from the classic SAT‑to‑bounded‑memory‑strategy problem, while membership in NP follows from the nondeterministic synthesis algorithm.

The paper also discusses a refined memory model called “compliance memory”. In this model, the memory update function is defined only for transitions that follow the prescribed action; when a deviation occurs, the system records the deviator and switches to a memoryless punishment strategy. This captures realistic scenarios where a system needs only to remember whether everyone has complied, not the full history of actions.

From a practical standpoint, the results suggest that automated synthesis tools can generate optimal leader strategies for embedded controllers, network protocols, or any setting where a central authority (the leader) can enforce policies but must operate with limited onboard memory. The NP‑completeness result indicates that, while the synthesis problem is computationally hard in the worst case, the nondeterministic algorithm provides a basis for heuristic or SAT‑based implementations that can handle realistic instances.

In summary, the paper extends the theory of discounted sum games to the bounded‑memory setting, establishes the existence and optimality of leader equilibria under memory constraints, provides a concrete synthesis method, demonstrates that more memory can strictly improve outcomes, and fully characterises the associated decision problem as NP‑complete. This bridges a gap between abstract game‑theoretic solutions and their implementation in resource‑constrained systems.