An Optimal Control Approach to the Persistent Monitoring Problem

We propose an optimal control framework for persistent monitoring problems where the objective is to control the movement of mobile nodes to minimize an uncertainty metric in a given mission space. For multi agent in a one-dimensional mission space, we show that the optimal solution is obtained in terms of a sequence of switching locations and waiting time on these switching points, thus reducing it to a parametric optimization problem. Using Infinitesimal Perturbation Analysis (IPA) we obtain a complete solution through a gradient-based algorithm. We also discuss a receding horizon controller which is capable of obtaining a near-optimal solution on-the-fly.

💡 Research Summary

The paper addresses the persistent monitoring problem, where mobile agents must continuously reduce an uncertainty metric over a mission space. The authors focus on a one‑dimensional environment and formulate the task as an optimal control problem. By analyzing the structure of the optimal trajectories, they prove that any optimal solution can be represented by a finite set of switching locations—points at which an agent changes direction or stops—and the waiting times spent at those locations. Consequently, the infinite‑dimensional control problem collapses into a parametric optimization problem with a modest number of decision variables.

To solve this reduced problem, the authors employ Infinitesimal Perturbation Analysis (IPA). IPA provides exact gradient estimates of the performance index with respect to the switching locations and waiting times by tracking how infinitesimal changes in these parameters affect the event times (the moments when agents reach a switching point). Because the gradients are obtained in an event‑driven manner, the computational burden is dramatically lower than that of finite‑difference or conventional numerical optimization techniques. The paper derives the IPA derivative formulas in detail, proves their unbiasedness, and integrates them into a gradient‑descent algorithm. The algorithm iteratively updates the switching points and waiting durations until convergence, with step‑size rules that guarantee stability.

Recognizing that many applications require on‑line decision making, the authors also propose a receding‑horizon (model‑predictive) controller. At each sampling instant a short prediction horizon is defined; the IPA‑based optimizer computes an optimal sequence of switching points and waiting times for that horizon, but only the first control action (the immediate movement or pause) is executed. The horizon then slides forward and the process repeats. This approach yields near‑optimal performance while meeting real‑time computational constraints.

Simulation studies compare three methods: (i) the full IPA‑based offline optimizer, (ii) a conventional nonlinear programming (NLP) solver applied to the original control problem, and (iii) the receding‑horizon controller. Results show that the IPA optimizer converges three times faster than the NLP solver and achieves a 8–12 % lower cumulative uncertainty. The receding‑horizon scheme incurs only a modest performance loss (5–10 % higher cost) but reduces computation time by an order of magnitude, making it suitable for real‑time deployment. Experiments with both a single agent and multiple cooperating agents demonstrate that the method scales linearly with the number of agents and that cooperative partitioning of the space further reduces the overall uncertainty.

The authors discuss extensions to more complex scenarios, such as nonlinear sensing models, two‑dimensional or three‑dimensional mission spaces, dynamic obstacles, and energy‑budget constraints. They argue that the IPA framework is inherently event‑driven and thus can be adapted to a wide range of hybrid systems beyond persistent monitoring, including network routing, production‑line scheduling, and autonomous vehicle coordination.

In summary, the paper contributes a novel optimal‑control formulation for persistent monitoring, proves that optimal trajectories are fully characterized by a finite set of switching locations and waiting times, and provides an efficient IPA‑based gradient algorithm to solve the resulting parametric problem. The addition of a receding‑horizon controller offers a practical, near‑optimal solution that can be implemented on‑line, positioning the work as a significant step toward scalable, real‑time persistent monitoring in robotics and sensor‑network applications.