Past-Discounting is Key for Learning Markovian Fairness with Long Horizons

Fairness is an important consideration for dynamic resource allocation in multi-agent systems. Many existing methods treat fairness as a one-shot problem without considering temporal dynamics, which misses the nuances of accumulating inequalities over time. Recent approaches overcome this limitation by tracking allocations over time, assuming perfect recall of all past utilities. While the former neglects long-term equity, the latter introduces a critical challenge: the augmented state space required to track cumulative utilities grows unboundedly with time, hindering the scalability and convergence of learning algorithms. Motivated by behavioral insights that human fairness judgments discount distant events, we introduce a framework for temporal fairness that incorporates past-discounting into the learning problem. This approach offers a principled interpolation between instantaneous and perfect-recall fairness. Our central contribution is a past-discounted framework for memory tracking and a theoretical analysis of fairness memories, showing past-discounting guarantees a bounded, horizon-independent state space, a property that we prove perfect-recall methods lack. This result unlocks the ability to learn fair policies tractably over arbitrarily long horizons. We formalize this framework, demonstrate its necessity with experiments showing that perfect recall fails where past-discounting succeeds, and provide a clear path toward building scalable and equitable resource allocation systems.

💡 Research Summary

Paper Overview
The authors address a fundamental scalability problem in temporal fairness for dynamic multi‑agent resource allocation. Existing approaches either ignore the history of allocations (instantaneous fairness) or retain the full history (perfect‑recall fairness). While perfect‑recall can in principle guarantee equitable outcomes over long horizons, it requires augmenting the state with a cumulative utility vector that grows without bound as the horizon increases. This unbounded state space makes reinforcement‑learning (RL) algorithms computationally intractable and prevents convergence in long‑running tasks.

Key Insight
Drawing on behavioral economics and moral psychology, the authors note that humans naturally discount events that occurred further in the past. They translate this insight into a formal “past‑discounted” memory mechanism: each agent’s accumulated utility is multiplied by a discount factor γₚ∈