Line-Search Filter Differential Dynamic Programming for Optimal Control with Nonlinear Equality Constraints

We present FilterDDP, a differential dynamic programming algorithm for solving discrete-time, optimal control problems (OCPs) with nonlinear equality constraints. Unlike prior methods based on merit functions or the augmented Lagrangian class of algorithms, FilterDDP uses a step filter in conjunction with a line search to handle equality constraints. We identify two important design choices for the step filter criteria which lead to robust numerical performance: 1) we use the Lagrangian instead of the cost in the step acceptance criterion and, 2) in the backward pass, we perturb the value function Hessian. Both choices are rigorously justified, for 2) in particular by a formal proof of local quadratic convergence. In addition to providing a primal-dual interior point extension for handling OCPs with both equality and inequality constraints, we validate FilterDDP on three contact implicit trajectory optimisation problems which arise in robotics.

💡 Research Summary

This paper introduces FilterDDP, a novel differential dynamic programming (DDP) algorithm designed to solve discrete-time optimal control problems (OCPs) with nonlinear equality constraints. The core innovation lies in adopting a line-search filter method, a strategy with strong convergence guarantees used in nonlinear programming solvers like IPOPT, instead of the more common augmented Lagrangian (AL) or merit function approaches prevalent in prior constrained DDP works.

The algorithm operates through alternating backward and forward passes. The backward pass computes a perturbed Newton step for solving the local optimality conditions of the constrained Bellman subproblems. A key design choice is the perturbation of the KKT matrix (Equation 12) by adding regularization terms (δ_w I, -δ_c I). This ensures the matrix has the correct inertia, which is crucial for numerical stability and the theoretical proof of local quadratic convergence. From this step, feedforward gains (α_t) and feedback policies (β_t), as well as updates for the Lagrange multipliers (ψ_t, ω_t), are derived.

In the forward pass, a candidate trajectory is generated using these policies (Equation 14). Step acceptance is governed by a filter mechanism. The filter maintains a “taboo” region in the (θ, L) plane, where θ is the sum of constraint violations and L is the Lagrangian value. A trial step is accepted if it provides sufficient reduction in either θ or L (Equation 16). Under specific conditions (Equation 17), the algorithm switches to an Armijo-type condition on the Lagrangian alone (Equation 19). The paper provides a rigorous justification for using the Lagrangian in the filter criteria by proving that the quantity ’m’ (Equation 18) is precisely the directional derivative of the Lagrangian along the search direction (Corollary 1).

Another significant theoretical contribution is a formal proof of local quadratic convergence for FilterDDP, generalizing a previous result for unconstrained scalar DDP to the constrained, vector-valued case. The perturbation of the Hessian in the backward pass is central to this proof.

Beyond equality constraints, the paper outlines a primal-dual interior-point extension to FilterDDP for handling inequality constraints as well. The algorithm is implemented in Julia and evaluated on three challenging, contact-implicit robotic trajectory optimization problems: a cart-pole swing-up with Coulomb friction, an acrobot swing-up with joint limits, and a non-prehensile pushing task. Experimental results demonstrate that FilterDDP requires fewer iterations and is more robust than state-of-the-art AL-based constrained DDP methods, while being faster and more suitable for embedded systems compared to general-purpose NLP solvers like IPOPT. Ablation studies confirm the importance of the two main design choices: using the Lagrangian in the filter and perturbing the value function Hessian.