Primal-dual dynamical systems with closed-loop control for convex optimization in continuous and discrete time

Let H and G be real Hilbert spaces equipped with inner product •, • and norm • . In this paper, we consider the convex optimization problem with linear equality constraints of the form

where f : H → R is a differentiable convex function with L-Lipschitz continuous gradient for L > 0, A : H → G is a continuous linear operator and b ∈ G. The optimization problem of the form (1) arises in many applications across fields such as image recovery, machine learning, and network optimization, see [1][2][3][4] and the references therein.

The Lagrangian function associated with Problem (1) is defined as

where λ ∈ G is the Lagrange multiplier. A pair (x * , λ * ) ∈ H × G is said to be a saddle point of the Lagrangian function L iff

In the sequel, the set of saddle points of L is denoted by S. The set of feasible points of Problem ( 1) is denoted by F := {x ∈ H|Ax = b}. For any (x, λ) ∈ F × G, it holds that f (x) = L(x, λ). We assume that S = ∅. Let (x * , λ * ) ∈ S. Then, (x * , λ * ) ∈ S ⇔ 0 = ∇ x L(x * , λ * ) = ∇f (x * ) + A * λ * ,

where A * : G → H denotes the adjoint operator of A.

To solve Problem (1), in this paper, we propose the following “second-order primal” + “first-order dual” dynamical system whose damping is a feedback control of the gradient of the Lagrangian function:

where q > 0, p ≥ 1, t ≥ t 0 > 0 and τ : [t 0 , +∞) → (0, +∞) is the time scaling function which is non-decreasing and continuously differentiable. We obtain the fast convergence rates for the primal-dual gap, the feasibility violation, and the objective residual along the trajectory generated by System (2). Subsequently, we propose an autonomous primal-dual algorithm by time discretization of System (2), and give some convergence analysis.

Over the past decades, a series of studies have explored second-order dynamical systems in continuous and discrete time for the unconstrained convex optimization problem

Su et al. [5] proposed the following second-order dynamical system:

where α ≥ 3, and obtained the O1 t 2 convergence rate of the objective residual along the trajectory generated by the system (4). The system (4) has further been studied in several papers, including [6][7][8]. They showed that the objective residual converges at a rate of order o 1 t 2 for α > 3, and the trajectory weakly converges to the global minimizer of the objective function. In order to improve the convergence rate, Attouch et al. [9] proposed the system (4) with the time scaling:

where α ≥ 1 and β : [t 0 , +∞) → (0, +∞) is the time scaling function which is non-decreasing and continuously differentiable. They obtained the O On the other hand, primal-dual dynamics methods were also attracted an increasing interest by many researchers for solving the linear equality constrained convex optimization problem (1). Over the past few years, there have been a wide variety of works devoted to “second-order primal” + “second-order dual” continuous-time dynamical systems for solving Problem (1), see [10][11][12][13]. By time discretization of continuous-time dynamical systems, new accelerated primal-dual algorithms for solving Problem (1) have been proposed in [14][15][16][17]. Since the computational cost of a primal-dual method primarily arises from the sub-problem involving the primal variable, and a first-order dynamical system is simpler and more tractable to solve than a second-order one, He et al. [18] proposed the following “second-order primal” + “first-order dual” dynamical system for solving Problem (1):

where γ > 0 and δ > 0. This system can be viewed as an extension of Polyaks heavy ball with friction system in [19]. Note that “second-order primal” + “first-order dual” dynamical systems and discrete accelerated algorithms for solving Problem (1) have been studied in [20][21][22][23].

The present study is motivated by prior works in two domains: dynamical systems with Hessian-driven damping and those with closed-loop control. We now provide a comprehensive review of these research areas.

It is worth noting that the incorporation of Hessian-driven damping into the dynamical system has been a significant milestone in optimization and mechanics, as Hessian-driven damping can effectively mitigate the oscillations. Building on this foundation, many researchers use dynamical systems with Hessian-driven damping for solving the problem (3). Alvarez et al. [24] first proposed the second-order dynamical system with constant viscous damping and constant Hessian-driven damping:

where b = 1. They obtained some new convergence properties of the solution trajectory generated by the system (5). Attouch et al. [25] established the system ( 5) with a = α t , and obtained the O 1 t 2 convergence rate of the objective residual for solving the problem (3). To further accelerate the convergence rate, Attouch et al. [26] proposed the system (5) with a = α t and b = β(t), and obtained the O

convergence rate for the objective residual. Moreover, for lead