Frequency Theorem for discrete time stochastic system with multiplicative noise

Kalman-Yakubovich Lemma (KY Lemma) was a groundbreaking result that paved way for a solutions to lots of problems in control theory, including optimal control. The first variant of the Lemma was derived by Yakubovich in 1962 (see [22]). The following year, the discrete-time version of that result was derived by Szegő and Kalman (see [20]). It is called sometimes the Kalman-Szegő Lemma (KS Lemma); see [5,13,17] for a comprehensive review of various results in control theory derived from the KY Lemma. Various works such as [1][2][3][4] considered problems with quadratic functionals whereas Yakubovich (see [25,26]) derived the KY Lemma for the case in which both the control and state vectors are both Hilbert spaces. Dokuchaev [6] considered a continuous time stochastic linear-quadratic optimal control problem, with the state evolution described by Itô equations, with state dependent coefficients; a generalization of the Frequency Theorem was obtained. We consider a discrete-time analogy of the problem studied in [6]. We show that the necessary and sufficient conditions for the existence of the optimal control can be formulated as matrix inequalities in frequency domain.

Furthermore, we show that if the optimal control exists, then certain Lyapunov equations must have a solution. The optimal control is obtained by solving a deterministic linear-quadratic optimal control problem whose functional depends on the solution to the Lyapunov equations.

Moreover, we show that under certain conditions, solvability of the Lyapunov equations is guaranteed. We also show that, if the frequency inequalities are strict, then the solution is unique up to equivalence.

We consider the following optimization problem on a standard probability space, (Ω, F , P).

over the set

subject to

and Γ = Γ ⊤ ∈ m × m are constant. The scalar ξ t ∈ R is the discrete-time white noise adapted to a flow of non-decreasing σ-algebras F t ⊂ F such that Eξ t = 0, Var (ξ t ) = 1. The vector a is random, measurable with respect to F 0 , independent of {ξ t } +∞ t=0 and is such that E |a| 2 < +∞ and E aa ⊤ 2 < +∞; we denote by |.| the Euclidean norm for vectors and Frobenius norm for matrices.

We assume all the matrices in (2.2) and (2.3) are real and we restrict our considerations to the case when all eigenvalues λ (A) of A lie inside the unit disk on the complex plane (that is, the spectral radius of A is ρ (A) < 1). Moreover, we assume that the system is stable in mean-square sense for u t ≡ 0. Various sufficient conditions of this stability can be found in [7-12, 15, 17, 18, 21] and other works.

For random x t , y t ∈ C n we denote the inner product (x . , y . ) by (x . , y . ) = +∞ t=0 E x ⊤ t y t and the norm by x . = (x . , x . ). Furthermore, we write

Condition 3.1. There exist symmetric matrices H and Θ in C n×n satisfying

Let Θ be the matrix satisfying Condition 3.1. Consider the hermitian form F :

given by

Let g : C → C n×n be the matrix-valued function

We denote the unit circle by ζ = {z ∈ C : |z| = 1}.

The following Theorem establishes necessary and sufficient conditions for the existence of optimal u o for the problem (2.1)-(2.4)

ii) Furthermore, if there exists a δ > 0 such that

then u o is unique (up to equivalence).

Theorem 3.1 above is an analog of KS Lemma for discrete-time optimal stochastic control problem (2.1)-(2.4). This is a discrete time version of a continuous-time result obtained in [6] for the case when γ = 0 and in Chapter 5 of [16]) for the general γ.

3.1 Proof of Theorem 3.1

Proof. Let

From (2.3)-(2.4) and (3.11), we have

Thus, using the fact that u t ∈ U and ρ (A) < 1, it follows from (3.13)-(3.14) that µ t < +∞. From (2.3)-(2.4) and (3.12), we have

Let us denote the j-th colum of a matrix D by D (j) .

We define the vectors q t , m t ∈ C n 2 as

The vectors q t and m t are formed by stacking up the columns of the matrices Q t and M t , respectively. Set A = A ⊗ A + C ⊗ C (where ⊗ denotes the Kronecker product). We can then rewrite (3.15) as

Note that the system in (3.18) is of dimension n 2 , however, due symmetry, it can be reduced to a system of dimension

The assumption that the system (2.3)-(2.4) is stable in the mean-square sense for u t = 0, is equivalent to m t being stable for q t = 0, which is true if and only if the spectral radius of A is ρ (A) < 1. From the solution of (3.18), we can show, using Hölder’s inequality and Young’s theorem, that m t 1 < +∞, therefore sup t≥0 E |x t | 2 < +∞. This compoletes the proof of Lemma 3.1.

It follows from Lemma 3.1 that the Z-transform, x (z), of x t , exists, and it’s radius of convergence contains the unit circle, ζ. If we set x t = 0, u t = 0 for all t < 0 we can then take the Z-transform of the system (2.3)-(2.4) and obtain

Let D be an n × n real symmetric matrix and let T → R n×n × R n×n be defined by

Please notice that the system in (3.23) would be degenerate if and only if A 1 = A 2 ; however, this would require that A = A ⊗ A + C ⊗ C = I n

…(Full text truncated)…

This content is AI-processed based on ArXiv data.