In this paper we provide optimal bounds for fully discrete approximations to finite horizon problems via dynamic programming. We adapt the error analysis in \cite{nos} for the infinite horizon case to the finite horizon case. We prove an a priori bound of size $O(h+k)$ for the method, $h$ being the time discretization step and $k$ the spatial mesh size. Arguing with piecewise constants controls we are able to obtain first order of convergence in time and space under standard regularity assumptions, avoiding the more restrictive regularity assumptions on the controls required in \cite{nos}. We show that the loss in the rate of convergence in time of the infinite case (obtained arguing with piece-wise controls) can be avoided in the finite horizon case
The numerical approximation of optimal control problems is of importance for many applications. In this paper we consider the dynamic programming approach for solving finite horizon problems. The dynamic programming principle gives a characterization of the value function as the unique viscosity solution of a nonlinear partial differential equation, the Hamilton-Jacobi-Bellman (HJB) equation. The value function is then used to get a synthesis of a feedback control law.
In the present paper our concern is to give optimal error bounds for a fully discrete semi-Lagrangian method approaching the value function. For a method with a positive time step size h and spatial elements of size k we prove an optimal error bound of size O(h + k) which gives first order of convergence in time and space for the method. We introduce a characterization of the fully discrete method inherited from [9], see also [10]. The temporal component of the error comes from the approximation of the dynamics by a discrete one based on the Euler method plus the approximation of the time integral by the composite rectangle rule. The spatial component of the error comes from the substitution of the functions, both in the dynamics and in the cost, by piece-wise linear interpolants in space. Adapting the technique in [9, Section 3.2], based on piece-wise constant controls in time, we avoid making regularity assumptions on the controls. We only need a kind of discrete regularity assumption on the computed discrete controls to achieve the full order of convergence, see Remark 1.
We think that the error analysis techniques shown in this paper are of interest to analyze similar methods applied to the same or analogous problems. To our knowledge, there are not many papers getting error estimates for methods solving finite horizon control problems. In [11], a dynamic programming algorithm based on a tree structure (which does not require a spatial discretization of the problem) to mitigate the curse of dimensionality, see [1], is analyzed. First order bounds in time are obtained in the first part of this paper. In the second part, it is assumed that the continuous set of controls is replaced by a discrete set. The tree structure considers only spatial nodes that result of the discrete dynamics. To reduce the increase in the total number of nodes a pruning criterion is applied in [1] that replace a new node by and old one whenever the distance between them is small enough. In [1] the pruning condition is too demanding since the difference between nodes is taken O(h 2 ), h being the time step. This fact comes from a factor h that appears dividing in the error bounds. This problem can be solved with the error analysis presented in the present paper. The same problem was fixed with the error analysis of [9] in the infinite horizon case. Reference [9] is the first one in which a rate of convergence O(h + k) is proved, improving the rate of convergence of size O(k/h) shown in the literature, see [6,Corollary 2.4], [7,Theorem 1.3]).
Although the method analyzed in this paper does not avoid the curse of dimensionality one can apply reduced order techniques to this method following [8]. In [8] a reduced order method based on proper orthogonal decomposition (POD) is applied for the numerical approximation of infinite horizon optimal control problems. The same ideas extend to finite horizon problems and will be subject of future research. For the error analysis of a reduced order method the error analysis of the present paper is essential. Moreover, as stated before, the error analysis shown in this paper can be used to analyze or improve the analysis of numerical methods for the same problem.
The outline of the paper is as follows. In Section 2 we introduce some notation. The fully discrete approximation is described in Section 3. In Section 4 we carry out the error analysis of the method. Finally, some interpolation arguments needed for the proof of the main theorem are included in the appendix.
Throughout this section we follow the notation in [1]. Let us consider the system ẏ(s) = f (y(s), u(s), s), s ∈ (t, T ],
(1)
We will denote by y : [t, T ] → R d the solution, by u the control u : [t, T ] → R m , by f : R d × R m → R d the dynamics, and by
the set of admissible controls where U ⊂ R m is a compact set. We assume that there exists a unique solution for (1) for each u ∈ U . The cost functional for the finite horizon optimal control problem will be given by
where
and λ ≥ 0 is the discount factor. The goal is to find a state-feedback control law u(t) = Φ(y(t), t), in terms of the state variable y(t), where Φ is the feedback map. To derive optimality conditions, dynamic programming principle (DPP) is used. The value function for an initial condition is defined by v(x, t) := inf u∈U J x,t (u).
(
The value function (3) satisfies the HHB equation for every x ∈ R d , s ∈ [t, T ):
If the value function is known, then it is possible to compute the opt
This content is AI-processed based on open access ArXiv data.