The $n$-step delayed sharing information structure is investigated. This information structure comprises of $K$ controllers that share their information with a delay of $n$ time steps. This information structure is a link between the classical information structure, where information is shared perfectly between the controllers, and a non-classical information structure, where there is no "lateral" sharing of information among the controllers. Structural results for optimal control strategies for systems with such information structures are presented. A sequential methodology for finding the optimal strategies is also derived. The solution approach provides an insight for identifying structural results and sequential decomposition for general decentralized stochastic control problems.
One of the difficulties in optimal design of decentralized control systems is handling the increase of data at the control stations with time. This increase in data means that the domain of control laws increases with time which, in turn, creates two difficulties. Firstly, the number of control strategies increases doubly exponentially with time; this makes it harder to search for an optimal strategy. Secondly, even if an optimal strategy is found, implementing functions with time increasing domain is difficult.
In centralized stochastic control [1], these difficulties can be circumvented by using the conditional probability of the state given the data available at the control station as a sufficient statistic (where the data available to a control station comprises of all observations and control actions till the current time) . This conditional probability, called information state, takes values in a time-invariant space. Consequently, we can restrict attention to control laws with time-invariant domain. Such results, in which data that is increasing with time is “compressed” to a sufficient statistic taking values in a time-invariant space, are called structural results. While the information state and structural result for centralized stochastic control problems are well known, no general methodology to find such information states or structural results exists for decentralized stochastic control problems.
The structural results in centralized stochastic control are related to the concept of separation. In centralized stochastic control, the information state, which is conditional probability of the state given all the available data, does not depend on the control strategy (which is the collection of control laws used at different time instants). This has been called a one-way separation between estimation and control. An important consequence of this separation is that for any given choice of control laws till time t -1 and a given realization of the system variables till time t, the information states at future times do not depend on the choice of the control law at time t but only on the realization of control action at time t. Thus, the future information states are separated from the choice of the current control law. This fact is crucial for the formulation of the classical dynamic program where at each step the optimization problem is to find the best control action for a given realization of the information state. No analogous separation results are known for general decentralized systems.
In this paper, we find structural results for decentralized control systems with delayed sharing information structures. In a system with n-step delayed sharing, every control station knows the n-step prior observations and control actions of all other control stations. This information structure, proposed by Witsenhausen in [2], is a link between the classical information structures, where information is shared perfectly among the controllers, and the non-classical information structures, where there is no “lateral” sharing of information among the controllers. In his seminal paper [2], Witsenhausen asserted a structural result for this model without any proof. Varaiya and Walrand [3] proved that Witsenhausen’s assertion was true for n = 1 but false for n > 1. For n > 1, Kurtaran [4] proposed another structural result. However, Kurtaran proved his result only for the terminal time step (that is, the last time step in a finite horizon problem); for non-terminal time steps, he gave an abbreviated argument, which we believe is incomplete. (The details are given in Section 5 of the paper).
We prove two structural results of the optimal control laws for the delayed sharing information structure. We compare our results to those conjectured by Witsenhausen and show that our structural results for n-step delay sharing information structure simplify to that of Witsenhausen for n = 1; for n > 1, our results are different from the result proposed by Kurtaran. Our structural results do not have the separated nature of centralized stochastic control: for any given realization of the system variables till time t, the realization of information states at future times depend on the choice of the control law at time t. However, our second structural result shows that this dependence only propagates to the next n -1 time steps. Thus, the information states from time t + n -1 onwards are separated from the choice of control laws before time t; they only depend on the realization of control actions at time t. We call this a delayed separation between information states and control laws.
The absence of classical separation rules out the possibility of a classical dynamic program to find the optimum control laws. However, optimal control laws can still be found in a sequential manner. Based on the two structural results, we present two sequential methodologies to find optimal control laws. Unlike classical dynamic programs,
This content is AI-processed based on open access ArXiv data.
