Minimization of entropy functionals

1. Introduction 1.1. The entropy minimization problem. Let R be a positive measure on a space Z. Take a [0, ∞]-valued measurable function γ * on Z × R such that γ * (z, •) := γ * z is convex and lower semicontinuous for all z ∈ Z. Denote M Z the space of all signed measures Q on Z. The entropy functional to be considered is defined by

where Q ≺ R means that Q is absolutely continuous with respect to R. Assume that for each z there exists a unique m(z) which minimizes γ * z with γ * z (m(z)) = 0, ∀z ∈ Z.

(1.2)

Then, I is [0, ∞]-valued, its unique minimizer is mR and I(mR) = 0. This paper is concerned with the minimization problem

where T o : M Z → X o is a linear operator which takes its values in a vector space X o and C is a convex subset of X o .

1.2. Presentation of the results. Our aim is to reduce as much as possible the restrictions on the convex set C. Denoting the minimizer Q of (1.3), the geometric picture is that some level set of I is tangent at Q to the constraint set T -1 o C. Since these sets are convex, they are separated by some affine hyperplane and the analytic description of this separation yields the characterization of Q. Of course Hahn-Banach theorem is the key. Standard approaches require C to be open with respect to some given topology in order to be allowed to apply it. In the present paper, one chooses to use a topological structure which is designed for the level sets of I to “look like” open sets, so that Hahn-Banach theorem can be applied without assuming to much on C.

This strategy is implemented in [17] in an abstract setting suitable for several applications. It is a refinement of the standard saddle-point method [22] where convex conjugates play an important role. The proofs of the present article are applications of the general results of [17].

Clearly, for the problem (1.3) to be attained, T -1 o C must share a supporting hyperplane with some level set of I. This is the reason why it is assumed to be closed with respect to the above mentioned topological structure. This will be the only restriction to be kept together with the interior specification (1.4) below.

Dual equalities and primal attainment are obtained under the weakest possible assumption:

C ∩ T o dom I = ∅ where dom I := {Q ∈ M Z ; I(Q) < ∞} is the effective domain of I and T o dom I is its image by T o . The main result of this article is the characterization of the minimizers of (1.3) in the interior case which is specified by

where icor (T o dom I) is the intrinsic core of T o dom I. The notion of intrinsic core does not rely on any topology; it gives the largest possible interior set. For comparison, a usual form of constraint qualification required for the representation of the minimizers of (1.3) is int (C) ∩ T o dom I = ∅ (1.5) where int (C) is the interior of C with respect to a topology which is not directly connected to the “geometry” of I. In particular, int (C) must be nonempty; this is an important restriction. The constraint qualification (1.4) is weaker.

An extension of Problem (1.3) is also investigated. One considers an extension Ī of the entropy I to a vector space L Z which contains M Z and may also contain singular linear forms which are not σ-additive. The extended problem is

Even if I is strictly convex, Ī isn’t strictly convex in general so that (1.6) may admit several minimizers. There are situations where (1.3) is not attained in M Z while (1.6) is attained in L Z . Other relations between these minimization problems are investigated by the author in [18] with probabilistic questions in mind.

1.3. Literature about entropy minimization. Entropy minimization problems appear in many areas of applied mathematics and sciences. The literature about the minimization of entropy functionals under convex constraints is considerable: many papers are concerned with an engineering approach, working on the implementation of numerical procedures in specific situations. In fact, entropy minimization is a popular method to solve ill-posed inverse problems.

Rigorous general results on this topic are quite recent. Let us cite, among others, the main contribution of Borwein and Lewis: [1], [2], [3], [4], [5], [6] together with the paper [23] by Teboulle and Vajda. In these papers, topological constraint qualifications of the type of (1.5) are required. Such restrictions are removed here. With a geometric point of view, Csiszár [8,9] provides a complete treatment of (1.3) with the relative entropy (see Section 6.1) under the weak assumption (1.4). The behavior of minimizing sequences of general entropy functionals is studied in [10].

By means of a method different from the saddle-point approach, the author has already studied in [15,16] entropy minimization problems under affine constraints (corresponding to C reduced to a single point) and more restrictive assumptions on γ * . The present article extends these results.

Outline of the paper. The minimization problems (1.3) and (1.6) are desc

This content is AI-processed based on open access ArXiv data.