Critical dynamics in the evolution of stochastic strategies for the iterated Prisoners Dilemma

The observed cooperation on the level of genes, cells, tissues, and individuals has been the object of intense study by evolutionary biologists, mainly because cooperation often flourishes in biological systems in apparent contradiction to the selfish goal of survival inherent in Darwinian evolution. In order to resolve this paradox, evolutionary game theory has focused on the Prisoner’s Dilemma (PD), which incorporates the essence of this conflict. Here, we encode strategies for the iterated Prisoner’s Dilemma (IPD) in terms of conditional probabilities that represent the response of decision pathways given previous plays. We find that if these stochastic strategies are encoded as genes that undergo Darwinian evolution, the environmental conditions that the strategies are adapting to determine the fixed point of the evolutionary trajectory, which could be either cooperation or defection. A transition between cooperative and defective attractors occurs as a function of different parameters such a mutation rate, replacement rate, and memory, all of which affect a player’s ability to predict an opponent’s behavior.

💡 Research Summary

**
The paper investigates how cooperation can evolve and persist in populations when strategies for the iterated Prisoner’s Dilemma (IPD) are encoded as mutable genetic traits. Each player is represented by five “genes”: an unconditional probability to cooperate on the first encounter (P C) and four conditional probabilities (P XY) that determine the chance of cooperating after the previous round resulted in X for the focal player and Y for the opponent (X,Y∈{C,D}). These probabilities are continuous values between 0 and 1 and are subject to per‑gene mutation at rate µ, where a mutation replaces the gene with a new value drawn uniformly from the interval.

Simulations are performed on a 32 × 32 toroidal lattice (spatially structured) and on a well‑mixed population (every empty site can be filled by any individual). In each update every individual plays a single round of the IPD with each of its eight neighbours, using the standard Axelrod payoff matrix (T = 5, R = 3, P = 1, S = 0). After the interaction phase a fraction r of individuals is randomly removed (replacement rate). The vacant sites are refilled by offspring of neighbours (spatial case) or of any individual in the population (well‑mixed case), with parents chosen proportionally to accumulated payoff (fitness). The process is repeated for 500 000 updates.

For each run the authors trace a line of descent (LOD) by selecting a random individual at the final generation and following its ancestry back to the initial random genotype (all five probabilities set to 0.5). By averaging the genotypes over the latter half of the LOD across 80 independent runs they obtain a “consensus genotype” that characterises the evolutionary attractor for the given parameter set.

Key findings:

1. Cooperative and Defective Attractors – In spatially structured populations with low mutation (µ ≈ 0.5 %) and low replacement (r ≈ 1 %) the consensus genotype is highly cooperative: P C ≈ 0.65, P CC ≈ 0.99, P CD ≈ 0.23, P DC ≈ 0.32, P DD ≈ 0.45. This strategy almost always repeats cooperation after mutual cooperation, tolerates occasional defections, and is willing to return to cooperation after a defection. When µ or r are increased, the population converges to a defective attractor with all four conditional probabilities near 0.5 and very low P CD and P DC, meaning players essentially random‑walk between C and D without systematic cooperation.

2. Phase‑Transition‑Like Behaviour – By performing principal‑component analysis (PCA) on the average LOD, the authors map evolutionary trajectories onto a two‑dimensional strategy space. As µ is raised, trajectories jump from the cooperative region (labelled RC) to the defective region (RD) at a well‑defined critical µ. They quantify the transition with an order parameter
   \