Bounded Planning in Passive POMDPs

In Passive POMDPs actions do not affect the world state, but still incur costs. When the agent is bounded by information-processing constraints, it can only keep an approximation of the belief. We present a variational principle for the problem of maintaining the information which is most useful for minimizing the cost, and introduce an efficient and simple algorithm for finding an optimum.

💡 Research Summary

The paper tackles a specialized class of partially observable Markov decision processes (POMDPs) called “Passive POMDPs,” in which the agent’s actions do not influence the underlying world state but still incur a cost. This setting isolates the informational aspect of decision‑making: the agent must decide which pieces of belief about the hidden state to retain when its information‑processing capacity is bounded. Traditional POMDP solutions assume unlimited memory and compute, allowing the agent to maintain the full Bayesian belief state. In contrast, the authors consider a realistic scenario where the agent can only store a compressed representation of its belief, limited by a fixed number of bits per time step (or an equivalent information‑rate constraint).

To address this, the authors formulate a variational principle that explicitly trades off expected cumulative cost against the information rate required to represent the belief. They introduce a Lagrangian

\