The statistical restricted isometry property and the Wigner semicircle distribution of incoherent dictionaries

A dictionary D is simply a set of vectors (also called atoms) in H. The number of vectors in D can exceed the dimension of the Hilbert space H, in fact, the most interesting situation is when |D| ≫ p = dim H. In this set-up we define a resolution of the Hilbert space H via D, which is the morphism of vector spaces

given by Θ (f ) = ϕ∈D f (ϕ) ϕ, for every f ∈ C (D). A more concrete way to think of the morphism Θ is as a p × |D| matrix with the columns being the atoms in D.

In the last two decades [11], and in particular in recent years [3,4,5,6,7,8], resolutions of Hilbert spaces became an important tool in signal processing, in particular in the emerging theories of sparsity and compressive sensing.

A useful property of a resolution is the restricted isometry property (RIP for short) defined by Candès-Tao in [7]. Fix a natural number n ∈ N and a pair of positive real numbers δ 1 , δ 2 ∈ R >0 . Definition 1.1. A dictionary D satisfies the restricted isometry property with coefficients (δ 1 , δ 2 , n) if for every subset S ⊂ D such that |S| ≤ n we have

for every function f ∈ C (D) which is supported on the set S.

Equivalently, RIP can be formulated in terms of the spectral radius of the corresponding Gram operator. Let G (S) denote the composition Θ * S • Θ S with Θ S denoting the restriction of Θ to the subspace C S (D) ⊂ C (D) of functions supported on the set S. The dictionary D satisfies (δ 1 , δ 2 , n)-RIP if for every subset S ⊂ D such that |S| ≤ n we have

where Id S is the identity operator on C S (D).

It is known [2,8] that the RIP holds for random dictionaries. However, one would like to address the following problem [1,10,9,20,21,22,23,25,24,26,27]: Problem 1.2. Find deterministic construction of a dictionary D with |D| ≫ p which satisfies RIP with coefficients in the critical regime

for some constant 0 < α < 1.

Fix a positive real number µ ∈ R >0 . The following notion was introduced in [9,12] and was used to study similar problems in [26,27]:

In this article we will explore a general relation between RIP and incoherence. Our motivation comes from three examples of incoherent dictionaries which arise naturally in the setting of finite harmonic analysis:

• The first example [18,19], referred to as the Heisenberg dictionary D H , is constructed using the Heisenberg representation of the finite Heisenberg group H (F p ). The Heisenberg dictionary is of size approximately p 2 and its coherence coefficient is µ = 1. • The second example [15,16,17], which is referred to as the oscillator dictionary D O , is constructed using the Weil representation of the finite symplectic group SL 2 (F p ). The oscillator dictionary is of size approximately p 3 and its coherence coefficient is µ = 4. • The third example [15,16,17], referred to as the extended oscillator dictionary D EO , is constructed using the Heisenberg-Weil representation [28,13] of the finite Jacobi group, i.e., the semi-direct product J (F p ) = SL 2 (F p ) ⋉ H (F p ). The extended oscillator dictionary is of size approximately p 5 and its coherence coefficient is µ = 4.

The three examples of dictionaries we just described constitute reasonable candidates for solving Problem 1.2: They are large in the sense that |D| ≫ p, and empirical evidences suggest (see [1] for the case of D H ) that they might satisfy RIP with coefficients in the critical regime (1.1). We summarize this as follows:

Question: Do the dictionaries D H , D O and D EO satisfy the RIP with coefficients δ 1 , δ 2 ≪ 1 and n = α • p, for some 0 < α < 1?

In this article we formulate a relaxed statistical version of RIP, called statistical isometry property (SRIP for short) which holds for any incoherent dictionary D which is, in addition, a disjoint union of orthonormal bases:

where

x is an orthonormal basis of H, for every x ∈ X.

The statistical isometry property. Let D be an incoherent dictionary of the form (3.1). Roughly, the statement is that for S ⊂ D, |S| = n with n = p 1-ε , for 0 < ε < 1, chosen uniformly at random, the operator norm G (S) -Id S is small with high probability. Precisely, we have Theorem 3.1 (SRIP property [14]). For every k ∈ N, there exists a constant C (k) such that the probability

above theorem, in particular, implies that probability P G (S) -Id S ≥ p -ε/2 → 0 as p → ∞ faster then p -l for any l ∈ N.

The statistics of the eigenvalues. A natural thing to know is how the eigenvalues of the Gram operator G (S) fluctuate around 1. In this regard, we study the spectral statistics of the normalized error term

Let ρ E(S) = n -1 n i=1 δ λ i denote the spectral distribution of E (S) where λ i , i = 1, .., n, are the real eigenvalues of the Hermitian operator E (S). Th

…(Full text truncated)…