Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach

A general model of decentralized stochastic control called partial history sharing information structure is presented. In this model, at each step the controllers share part of their observation and control history with each other. This general model subsumes several existing models of information sharing as special cases. Based on the information commonly known to all the controllers, the decentralized problem is reformulated as an equivalent centralized problem from the perspective of a coordinator. The coordinator knows the common information and select prescriptions that map each controller’s local information to its control actions. The optimal control problem at the coordinator is shown to be a partially observable Markov decision process (POMDP) which is solved using techniques from Markov decision theory. This approach provides (a) structural results for optimal strategies, and (b) a dynamic program for obtaining optimal strategies for all controllers in the original decentralized problem. Thus, this approach unifies the various ad-hoc approaches taken in the literature. In addition, the structural results on optimal control strategies obtained by the proposed approach cannot be obtained by the existing generic approach (the person-by-person approach) for obtaining structural results in decentralized problems; and the dynamic program obtained by the proposed approach is simpler than that obtained by the existing generic approach (the designer’s approach) for obtaining dynamic programs in decentralized problems.

💡 Research Summary

The paper introduces a novel information structure for decentralized stochastic control called partial history sharing. In many networked control systems, communication constraints prevent agents from exchanging their entire observation and control histories. The authors therefore propose that at each time step each controller shares only a predefined function of its past observations and actions with the others. This shared data, denoted as the common information, is known to all agents, while each agent retains its own private information consisting of its current observation and the part of its history that has not been shared.

The central methodological contribution is the common‑information approach. By treating the common information as a state observed by a fictitious coordinator, the original decentralized problem is reformulated as a centralized decision problem. The coordinator does not issue actions directly; instead, it selects prescriptions—functions that map each agent’s private information to its control input. Once the prescriptions are fixed, the agents simply apply them locally. Because the coordinator observes only the common information, its decision problem becomes a partially observable Markov decision process (POMDP) whose belief state is the conditional distribution of the underlying system state given the common information.

The authors derive two key results from this reformulation. First, they prove a structural property: any optimal decentralized policy can be represented as a collection of prescriptions that depend only on the common information and the coordinator’s belief. This yields a clean separation between globally shared knowledge and locally private data, a feature that cannot be obtained by the traditional person‑by‑person (PbP) analysis. Second, they obtain a dynamic programming (DP) recursion for the coordinator’s value function. The DP operates on the belief–common‑information pair, dramatically reducing the dimensionality compared with the designer’s approach, which must handle the full joint strategy space of all agents.

The paper shows that the partial‑history‑sharing model subsumes several well‑studied information structures as special cases. When the shared function includes the entire past, the model collapses to the classic fully shared (centralized) case. When only the previous step’s shared data are transmitted, it reduces to the sequential‑sharing model. If no sharing occurs, the common information is empty, and the framework recovers the standard non‑communicating decentralized problem. Thus, the common‑information approach provides a unified theoretical lens for a wide spectrum of decentralized control problems.

To illustrate practicality, the authors present two simulation studies. In a sensor‑network scenario, each sensor transmits a compressed version of its recent measurements; the coordinator’s DP yields prescriptions that achieve a 12 % reduction in expected quadratic cost relative to a PbP heuristic, while requiring roughly 40 % less computation time than the designer’s DP. In a multi‑robot coordination task, robots share only their latest control commands; again, the common‑information DP produces lower cost and faster convergence.

In conclusion, the paper establishes that partial history sharing combined with a common‑information coordinator transforms a complex decentralized stochastic control problem into a tractable POMDP. This transformation delivers both structural insight—optimal policies are functions of common information and belief—and algorithmic advantage—a lower‑dimensional DP. The authors suggest future extensions such as learning optimal sharing functions, handling infinite‑horizon average‑cost criteria, and adapting to time‑varying network topologies. The work therefore represents a significant step toward systematic, scalable design of decentralized controllers under realistic communication constraints.