Multiplicative Weights Update with Constant Step-Size in Congestion Games: Convergence, Limit Cycles and Chaos

The Multiplicative Weights Update (MWU) method is a ubiquitous meta-algorithm that works as follows: A distribution is maintained on a certain set, and at each step the probability assigned to element $\gamma$ is multiplied by $(1 -\epsilon C(\gamma))>0$ where $C(\gamma)$ is the “cost” of element $\gamma$ and then rescaled to ensure that the new values form a distribution. We analyze MWU in congestion games where agents use \textit{arbitrary admissible constants} as learning rates $\epsilon$ and prove convergence to \textit{exact Nash equilibria}. Our proof leverages a novel connection between MWU and the Baum-Welch algorithm, the standard instantiation of the Expectation-Maximization (EM) algorithm for hidden Markov models (HMM). Interestingly, this convergence result does not carry over to the nearly homologous MWU variant where at each step the probability assigned to element $\gamma$ is multiplied by $(1 -\epsilon)^{C(\gamma)}$ even for the most innocuous case of two-agent, two-strategy load balancing games, where such dynamics can provably lead to limit cycles or even chaotic behavior.

💡 Research Summary

This paper conducts a comparative analysis of the dynamical behavior of two nearly homologous variants of the Multiplicative Weights Update (MWU) algorithm—the linear variant (MWU_ℓ) and the exponential variant (MWU_e)—within the context of atomic congestion games, when agents employ constant learning rates.

The central positive finding is a deterministic convergence guarantee for MWU_ℓ. The authors prove that in any congestion game, regardless of the number of agents, the structure of strategy sets, or even when agents use different constant learning rates ε_i (provided the update factors remain positive), the dynamics of MWU_ℓ always converge to a fixed point. Moreover, starting from interior points (where all strategies have positive probability), the convergence is to an exact Nash equilibrium of the game. The key technical innovation enabling this proof is the discovery of a novel connection between MWU_ℓ and the Baum-Eagon inequality. The authors show that the MWU_ℓ update rule in congestion games is a special case of the iterative process described by Baum and Eagon, which is known to strictly increase a certain homogeneous polynomial. By constructing an appropriate polynomial Q(p) related to the expected potential Ψ(p) = E_s∼p