Incoherent dictionaries and the statistical restricted isometry property

A dictionary D is simply a set of vectors (also called atoms) in H. The number of vectors in D can exceed the dimension of the Hilbert space H, in fact, the most interesting situation is when |D| ≫ p = dim H. In this set-up we define a resolution of the Hilbert space H via D, which is the morphism of vector spaces Θ :

given by Θ (f ) = ϕ∈D f (ϕ) ϕ, for every f ∈ C (D). A more concrete way to think of the morphism Θ is as a p × |D| matrix with the columns being the atoms in D.

In the last two decades [11], and in particular in recent years [3], [4], [5], [6], [7], [8], resolutions of Hilbert spaces became an important tool in signal processing, in particular in the emerging theories of sparsity and compressive sensing.

A useful property of a resolution is the restricted isometry property (RIP for short) defined by Candès-Tao in [7]. Fix a natural number n ∈ N and a pair of positive real numbers

Equivalently, RIP can be formulated in terms of the spectral radius of the corresponding Gram operator. Let G (S) denote the composition Θ * S • Θ S with Θ S denoting the restriction of Θ to the subspace C S (D) ⊂ C (D) of functions supported on the set S. The dictionary D satisfies (δ 1 , δ 2 , n)-RIP if for every subset S ⊂ D such that |S| ≤ n we have

where Id S is the identity operator on C S (D).

It is known [2], [8] that the RIP holds for random dictionaries. However, one would like to address the following problem [1], [10], [9], [20], [21], [22], [23], [25], [24], [26], [27]:

Problem II-.2: Find deterministic construction of a dictionary D with |D| ≫ p which satisfies RIP with coefficients in the critical regime

for some constant 0 < α < 1.

Fix a positive real number µ ∈ R >0 . The following notion was introduced in [9], [12] and was used to study similar problems in [26], [27]:

In this article we will explore a general relation between RIP and incoherence. Our motivation comes from three examples of incoherent dictionaries which arise naturally in the setting of finite harmonic analysis:

• The first example [18], [19], referred to as the Heisenberg dictionary D H , is constructed using the Heisenberg representation of the finite Heisenberg group H (F p ). The Heisenberg dictionary is of size approximately p 2 and its coherence coefficient is µ = 1. • The second example [15], [16], [17], which is referred to as the oscillator dictionary D O , is constructed using the Weil representation of the finite symplectic group SL 2 (F p ). The oscillator dictionary is of size approximately p 3 and its coherence coefficient is µ = 4. • The third example [15], [16], [17], referred to as the extended oscillator dictionary D EO , is constructed using the Heisenberg-Weil representation [28], [13] of the finite Jacobi group, i.e., the semi-direct product J (F p ) = SL 2 (F p ) ⋉ H (F p ). The extended oscillator dictionary is of size approximately p 5 and its coherence coefficient is µ = 4. The three examples of dictionaries we just described constitute reasonable candidates for solving Problem II-.2: They are large in the sense that |D| ≫ p, and empirical evidences suggest (see [1] for the case of D H ) that they might satisfy RIP with coefficients in the critical regime (II-.1). We summarize this as follows:

Problem III-.4: Do the dictionaries D H , D O and D EO satisfy the RIP with coefficients δ 1 , δ 2 ≪ 1 and n = α • p, for some 0 < α < 1?

In this article we formulate a relaxed statistical version of RIP, called statistical isometry property (SRIP for short) which holds for any incoherent dictionary D which is, in addition, a disjoint union of orthonormal bases:

where

x is an orthonormal basis of H, for every x ∈ X.

Let D be an incoherent dictionary of the form (IV-.2). Roughly, the statement is that for S ⊂ D, |S| = n with n = p 1-ε , for 0 < ε < 1, chosen uniformly at random, the operator norm G (S) -Id S is small with high probability. Precisely, we have Theorem IV-A.1 (SRIP property [14]): For every k ∈ N, there exists a constant C (k) such that the probability

The above theorem, in particular, implies that probability Pr G (S) -Id S ≥ p -ε/2 → 0 as p → ∞ faster then p -l for any l ∈ N.

A natural thing to know is how the eigenvalues of the Gram operator G (S) fluctuate around 1. In this regard, we study the spectral statistics of the normalized error term

Let ρ E(S) = n -1 n i=1 δ λ i denote the spectral distribution of E (S) where λ i , i = 1, .., n, are the real eigenvalues of the Hermitian operator E (S). The following theorem asserts that ρ E converges in probability as p → ∞ to the Wigner semicircle distribution ρ SC

Theorem IV-B.1 (Semicircle distribution [14]): We have

Remark IV-B.2: A limit of the form (IV-B.1

…(Full text truncated)…