Blind Multi-Band Signal Reconstruction: Compressed Sensing for Analog Signals

1 Blind Multi-Band Signal Reconstruction: Compressed Sensing for Analog Signals Moshe Mishali and Y onina C. Eldar , Member , IEEE Abstract W e add ress the pro blem of reconstruc ting a multi-band signal fr om its sub-Ny quist point-wise samples. T o date, all reco nstruction m ethods propo sed fo r this class of signals assume d knowledge of the band locatio ns. In this paper, we develop a non-linear blind perfect reconstructio n scheme for multi-ba nd sign als which does not require the band locations. Ou r appro ach assumes an existing blind mu lti-coset sampling method. The sp arse structu re of multi-band sign als in th e continuou s fr equency dom ain is used to replace th e continuous reconstruction with a single finite d imensional pr oblem witho ut the need for discretization . The resulting prob lem can be formu lated within the framew o rk of comp ressed sensing , an d thus can b e solved efficiently using known tra ctable algorithms from this e merging ar ea. W e also develop a theoretical lower bound on the average sampling rate required fo r blind signal reconstru ction, which is twice the min imal rate of known-spectrum recovery . Our method e nsures perfect recon struction fo r a wide class of signals sampled at the minim al rate. Numerical experiments ar e presented demonstra ting blind sampling and reconstru ction with min imal samplin g rate. Index T erms Kruskal-ran k, Land au-Nyqu ist rate, multiband , multiple measu rement vectors (MMV) , nonun iform period ic sampling, orthogo nal matchin g pur suit (OMP), signal re presentation, sparsity . I . I N T R O D U C T I O N The well kno wn Whittaker , K otel ´ nikov , and Shannon (WKS) theorem links an alog signa ls with a discrete representation, allowing the transfe r of the sign al proces sing to a digital framework. The the orem states that a real-valued signal bandlimited to B Hertz can be p erfectly rec onstructed from its un iform s amples if the sampling rate is a t least 2 B samples per second. This minimal rate is called the Nyquist rate of the signal. Multi-band signals are bandlimited s ignals that poss es an a dditional s tructure in the frequ ency domain. Th e spectral sup port of a mu lti-band signal is restricted to s ev e ral co ntinuous intervals. E ach of these interv als is called a band and it is as sumed that no information res ides outside the bands. The design of sa mpling and reconstruction systems for these signals in volves three major con siderations. One is the sampling ra te. T he o ther is the set of The authors are with the T echnion—Israel Institute of T echnology , Haifa Israel. Email: moshik o@tx.technion.ac.il, yon- ina@ee.technion.ac.il. 2 multi-band signals that the sy stem ca n perfectly reco nstruct. The last one is blindnes s, namely a design that d oes not a ssume kno wledge o f the band locations. Blindness is a desirable property as sign als with dif ferent band locations are proces sed in the same way . La ndau [1] developed a minimal sa mpling rate for an arbitrary samp ling method tha t allows perfect recons truction. For multi-band s ignals, the Lan dau rate is the sum of the ba nd widths, which is be low the correspon ding Nyq uist rate. Uniform sa mpling of a real ba ndpass signa l with a total width of 2 B Hertz on both s ides o f the spec trum was studied in [2]. It w a s sh own that only spe cial cas es o f ban dpass s ignals c an be perfec tly recon structed from their uniform samples at the minimal rate of 2 B samples/sec. K ohlenbe r g [3] suggested periodic n on-uniform s ampling with an average sampling rate of 2 B . He also provided a recons truction scheme that recovers any b andpas s signal exactly . Lin and V aidy anathan [4] extended his work to multi-band signals. Their method ensu res pe rfect reconstruction from p eriodic non uniform sampling with an average sampling rate equal to the Lan dau rate. Both of these works lack the blindn ess property as the information about the ba nd locations is used in the des ign of both the s ampling and the rec onstruction stages . Herley and W o ng [5] and V enkataramani and Bresler [8] suggested a blind mu lti-coset sampling strategy that is called u ni versa l in [8]. T he a uthors of [8] also developed a de tailed recon struction sc heme for this sampling strategy , which is no t blind as its des ign requ ires infor ma tion about the s pectral support of the signal. Blind multi-coset sampling renders the recons truction applicab le to a wide set of multi-band s ignals but not to all of them. Although spectrum-blind reconstruction was mentioned in two conferenc e papers in 1996 [6],[7], a full s pectrum- blind recon struction sc heme was not developed in the se pape rs. It a ppears that spectrum-blind rec onstruction h as not bee n ha ndled s ince then. W e begin by dev elop ing a lower bound o n the minimal sa mpling rate req uired for blind perfect recon struction with arbitrary sampling and reconstruction. As we show the lower bound is twice the Landau rate and no more than the Nyquist rate. This result is based on rec ent work of L ue an d Do [20] o n samp ling s ignals from a union of subs paces . The he art of this pape r is the development of a spectrum-blind rec onstruction (SBR) s cheme for multi-band signals. W e assume a blind multi-coset s ampling satisfying the minimal rate requ irement. Theoretical tools are developed in o rder to tr a nsform the continuou s n ature of t h e rec onstruction prob lem into a finite dimen sional problem without any discretization. W e then prove that the so lution can b e obtaine d by finding the unique s parsest solution matrix from Multi p le-Measurement-V e ctors (MMV). This set of operations is grouped und er a b lock we name Continuous to Fi n ite (CTF). This b lock is the c ornerstone of two SBR algorithms we dev e lop to reconstruct the s ignal. One is entitled S BR4 and enab les perfect recon struction using o nly on e instanc e of the CTF block but requires twice the minimal s ampling rate. The other is referred to a s SBR2 and allows for sampling a t the minimal rate, but in volves a bi-se ction p rocess and s ev era l uses of the CTF b lock. Other differences be tween the algorithms are also discus sed. Both SBR4 an d S BR2 can easily be implemented in DSP proces sors o r in software 3 en vironments. Our p roposed rec onstruction app roach is app licable to a broad class of multi-band signals. Th is c lass is the blind version of the set of s ignals considered in [8]. In pa rticular , we characterize a s ubset M of this class by the max imal number of bands and the width of the widest band. W e then s how h ow to choose the p arameters o f the multi-cose t stage so that pe rfect reconstruction is poss ible for every signa l in M . This parame ter selection is also valid for known-spectrum reconstruction with h alf the sa mpling rate. T he set M rep resents a natural cha racterization of multi-band s ignals base d on their intrinsic parameters which are usua lly k nown in ad vance. W e prove that the SBR4 algo rithm e nsures pe rfect reconstruction for all signals in M . The SBR2 approa ch works for almos t all signals in M but may fail in so me very special case s (which typically will n ot o ccur). As our strategy is applicable also for signa ls that do no t lie in M , we present a n ice feature of a s ucces s recovery indication. Thu s, if a signal cannot be re covered this indication prevents further proce ssing o f in valid data. The CTF block requires finding a spa rsest solution matri x which is an NP-hard problem [12]. Several sub-optimal efficient me thods h av e been developed for this p roblem in the compress ed s ensing (CS) literature [15],[16]. In our algorithms, any of the se techniques can be used. Numerical experiments on random con structions of multi-band signals show that bo th SBR4 and SBR2 ma intain a sa tisfactory exact recovery rate whe n the average sampling rate approache s their theo retical minimum rate requirement a nd s ub-optimal implementations of the CTF block are used. Mo reover , the a verage runtime is s hown to be f as t enoug h for prac tical us age. Our work differs f rom other main stream CS pa pers i n two aspects . The first i s that we aim t o rec over a continuous signal, w hile the c lassical problem ad dressed in the CS literature is the recovery of discrete and finite vectors. An adaptation of CS results to con tinuous s ignals was also considered in a set of co nferences p apers (see [21],[22] an d the referen ces therein). However , thes e papers did not a ddress the case o f multi-band signals. In [22] an u nderlying discrete mode l w a s assu med so that the sign al is a linear combination of a fi nite number of known func tions. Here, there is no disc rete mo del as the s ignals a re treated in a c ontinuous framework w ithout any discretization. The second aspe ct is that we a ssume a deterministic sampling stage a nd our theorems and results do no t in volve any probability mo del. In contrast, the co mmon approac h in comp ressiv e sensing assume s random sampling ope rators a nd typica l results are valid with s ome prob ability less than 1 [13],[19],[21],[22]. The pap er is organized as follo ws . In Section II we formulate our recons truction p roblem. The minimal dens ity theorem for blind reconstruction is stated and proved in Section III. A b rief overview of multi-coset sa mpling is presented in S ection IV. W e d ev e lop our main theoretical results on spectrum-blind reconstruction a nd present the CTF block in Section V. Based on these results, in Section VI , we design and comp are the SBR4 and the SBR2 algorithms. Numerical expe riments are des cribed in Section V II. 4 I I . P R E L I M I N A R I E S A N D P RO B L E M F O R M U L A T I O N A. No tation Common notation, as summarized in T able I , is used through out the paper . Exc eptions to this no tation are indicated in the te xt. T ABLE I N O TA T I O N x ( t ) continuous time signal with finite e nergy X ( f ) Fourier transform of x ( t ) (t h at is assumed to exist) a [ n ] bounde d ene r gy sequence z ∗ conjugate of the complex numbe r z ( ¯ v ) vector ( ¯ v ) i or ( ¯ v )( i ) i th en try of ( ¯ v ) ( ¯ v )( f ) vector tha t depen ds on a continuous p arameter f ( ¯ A ) matrix ( ¯ A ) ik ik th e ntry of ( ¯ A ) ( ¯ A ) T , ( ¯ A ) H transpose and the conjugate-transpo se of ( ¯ A ) ( ¯ A )  0 ( ¯ A ) is an H ermitian pos iti ve s emi-definite (PSD) matrix ( ¯ A ) † the Moore-Pen rose p seudo-in verse of ( ¯ A ) S finite or c ountable set S i i th eleme nt o f S | S | ca rdinality of a finite set S T infinite non-c ountable set λ ( T ) the Leb esgue me asure of T ⊆ R In addition, the following a bbreviations are use d. The ℓ p norm of a v e ctor ( ¯ v ) is de fined as k ( ¯ v ) k p p = X i | ( ¯ v ) i | p , p ≥ 0 . The d efault value for p is 2, s o that k ( ¯ v ) k d enotes the ℓ 2 norm of ( ¯ v ) . The stan dard L 2 norm is used for continuous signals. The i th column of ( ¯ A ) is writ ten as ( ¯ A ) i , the i th row is ( ( ¯ A ) T ) i written as a column vector . Indicator se ts for v e ctors and matrices a re d efined respec ti vely a s I ( ( ¯ v )) = { k | ( ¯ v )( k ) 6 = 0 } , I ( ( ¯ A )) = { k | ( ( ¯ A ) T ) k 6 = ( ¯ 0) } . The se t I ( ( ¯ v )) contains the indices of non -zero v a lues in the vector ( ¯ v ) . Th e set I ( ( ¯ A )) co ntains the indices o f the non -identical zero ro ws of ( ¯ A ) . Finally , ( ¯ A ) S is the ma trix that co ntains the co lumns of ( ¯ A ) with indice s belonging to the s et S . The matrix ( ¯ A ) S is referred to a s the (column s) restriction of ( ¯ A ) to S . F o rmally , ( ( ¯ A ) S ) i = ( ( ¯ A )) S i , 1 ≤ i ≤ | S | . 5 Similarly , ( ¯ A ) S is re ferred to as the rows r e striction of ( ¯ A ) to S . B. Multi-band signals In this work our prime focus is on the set M of all complex-valued mu lti-band signa ls ban dlimited to F = [0 , 1 /T ] with no more tha n N ba nds where ea ch of the b and widths is upper bou nded by B . Fig. 1 depicts a typical sp ectral s upport for x ( t ) ∈ M . Fig. 1. T ypical spectrum suppo rt of x ( t ) ∈ M . The Nyquist rate corresponding to any x ( t ) ∈ M is 1 /T . The Fourier transform o f a multi-band signa l ha s support on a finite union of disjoint interv als in F . Each interval is called a band a nd is uniquely represented by its edge s [ a i , b i ] . W ithout loss of generality it is assume d that the bands are n ot overlapping. Although our interes t is mainly in signals x ( t ) ∈ M , our resu lts are applicable to a broader c lass of signals, as explained in the relevant s ections. In addition, the results of the pape r are eas ily adopted to real-valued signals supported on [ − 1 / 2 T , +1 / 2 T ] . The required mod ifications are explained in Appen dix A an d are base d on the equations derived in Section IV -A. C. Problem formulation W e wish to perfectly reco nstruct x ( t ) ∈ M from its point-wise samples under two cons traints. On e is blindne ss, so that the information about the ba nd locations is not us ed wh ile acquiring the samples and neither ca n it be used in the reconstruction process . The other is that the samp ling rate req uired to guarantee perfect reconstruction should be minimal. This problem is solved if either of its cons traints is removed. W ithout the rate cons traint, the WKS theorem allows pe rfect blind-recons truction for every sign al x ( t ) bandlimited to F from its un iform samples at the Nyquist rate x ( t = n/T ) . Alternativ e ly , if the exact number of b ands and their locations are kno wn , the n the method of [4] allo ws perfect rec onstruction for every multi-band signa l at the minimal s ampling rate provided by Landa u’ s theorem [1]. In this pape r , we first dev elop the minimal sampling rate required for blind reconstruction. W e then use a multi- coset sampling strategy to acquire the s amples at an average sa mpling rate sa tisfying the minimal req uirement. The design of this s ampling method do es not re quire knowl e dge of the band locations. W e provide a spectrum-blind reconstruction sche me for this sa mpling strategy in the form of two different algorithms, name d SBR4 and SBR2 . It 6 is s hown that if the samp ling rate is twice the minimal rate then a lgorithm SBR4 gu arantees perfect reconstruction for every x ( t ) ∈ M . The SBR2 a lgorithm requ ires the minimal samp ling rate a nd g uarantees perfect recons truction for most signals in M . Howe ver , some special signals from M , discus sed in S ection VI-B, cannot be perfectly reconstructed by this ap proach. Excluding these special case s, our proposed method satisfies both constraints of the problem formulation. I I I . M I N I M A L S A M P L I N G R A T E W e begin b y quoting La ndau’ s theorem for the minimal sampling rate of an arbitrary sa mpling method that allows kno wn -spectrum perfect recons truction. It is then proved that blind pe rfect-reconstruction requires a minimal sampling rate tha t is twice the Landau rate. A. Kn own spectr um sup port Consider the space of ba ndlimited functions res tricted to a k nown support T ⊆ F : B T = { x ( t ) ∈ L 2 ( R ) | supp X ( f ) ⊆ T } . (1) A class ical sampling scheme takes the values of x ( t ) on a known co untable set of loca tions R = { r n } ∞ n = −∞ . The set R is called a samp ling set for B T if x ( t ) can be perfectly rec onstructed in a s table way from the s equenc e of samples x R [ n ] = x ( t = r n ) . The stability constraint requires the existence of constants α > 0 and β < ∞ s uch that: α k x − y k 2 ≤ k x R − y R k 2 ≤ β k x − y k 2 , ∀ x, y ∈ B T . (2) Landau [1] proved tha t if R is a sampling set for B T then it mus t have a density D − ( R ) ≥ λ ( T ) , wh ere D − ( R ) = lim r →∞ inf y ∈ R | R ∩ [ y , y + r ] | r (3) is the lo we r Beurling density , an d λ ( T ) is the Leb esgue meas ure of T . The numerator in (3) c ounts the number of points from R in every interval of width r of the real axis 1 . This result is us ually interpreted as a minimal average sampling rate req uirement for B T , and λ ( T ) is called the Lan dau rate . B. Un known spectru m supp ort Consider the set N Ω of signals ban dlimited to F with bandwidth o ccupation no more than 0 < Ω < 1 , so that λ (sup p X ( f )) ≤ Ω T , ∀ x ( t ) ∈ N Ω . 1 The numerator is not necessarily finite but as the sampling set is countable the infimum takes on a finite value. 7 The Nyquist rate for N Ω is 1 /T . N ote that N Ω is not a su bspace so that the Landau theorem is no t v alid he re. Nev e rtheless, it is intuiti ve to argue tha t the minimal samp ling rate for N Ω cannot be belo w Ω /T as this value is the Lan dau rate h ad the s pectrum su pport been kn own. A blind sampling set R for N Ω is a sampling set who se design d oes not as sume knowledge of sup p X ( f ) . Similarly to (2) the s tability o f R requires the existence of α > 0 and β < ∞ such tha t: α k x − y k 2 ≤ k x R − y R k 2 ≤ β k x − y k 2 , ∀ x, y ∈ N Ω . (4) Theorem 1 (Minimal sa mpling rate): Let R be a blind s ampling set for N Ω . Then , D − ( R ) ≥ m in  2Ω T , 1 T  . (5) Pr oof: The s et N Ω is of the form N Ω = [ T ∈ Γ B T , (6) where Γ = {T | T ⊆ F , λ ( T ) ≤ Ω /T } . (7) Clearly , N Ω is a non-cou ntable union of subsp aces. Sampling signals that lie in a union o f subs paces has be en recently treated in [20]. F o r every γ , θ ∈ Γ define the su bspace s B γ ,θ = B γ + B θ = { x + y | x ∈ B γ , y ∈ B θ } . (8) Since R is a sampling set for N Ω , (4) holds for some c onstants α > 0 , β < ∞ . It was proved in [20, P roposition 2] that (4) is valid if and o nly if α k x − y k 2 ≤ k x R − y R k 2 ≤ β k x − y k 2 , ∀ x, y ∈ B γ ,θ (9) holds for every γ , θ ∈ Γ . In pa rticular , R is a s ampling set for e very B γ ,θ with γ , θ ∈ Γ . Observe that the spac e B γ ,θ is of the form (1) with T = γ ∪ θ . Applying Landau ’ s de nsity theorem for eac h γ , θ ∈ Γ results in D − ( R ) ≥ λ ( γ ∪ θ ) , ∀ γ , θ ∈ Γ . (10) Choosing γ =  0 , Ω T  , θ =  1 − Ω T , 1 T  , we have that for Ω ≤ 0 . 5 , D − ( R ) ≥ λ ( γ ∪ θ ) = λ ( γ ) + λ ( θ ) = 2Ω T . (11) If Ω ≥ 0 . 5 then γ ∪ θ = F and D − ( R ) ≥ λ ( γ ∪ θ ) = 1 T . (12) 8 Combining (11) an d (12) completes the proof. In [20], the authors con sider minimal sampling requiremen ts for a union of shift-in vari a nt subs paces, with a particular s tructure of samp ling functions. Specifica lly , they view the samples as inn er prod ucts with sampling functions of the form { ψ k ( t − m ) } 1 ≤ k ≤ K,m ∈ Z , which inc ludes multi-coset samp ling. T heorem 1 extends this result to an arbitrary point-wise sampling operator . In particular , it is v a lid for non pe riodic sampling s ets that are not covered by [20]. An immediate corollary of Theorem 1 is that if Ω > 0 . 5 then uniform sampling at the Nyquist rate with a n ideal low pass filter satisfies the requirements of o ur problem formulation. Name ly , both the sa mpling and the reconstruction do not us e the information about the band loca tions, and the sampling rate is minimal ac cording to Theorem 1. As M is con tained in the spa ce of b andlimited signa ls, this ch oice als o provides perfect recon struction for every x ( t ) ∈ M . Therefore, in the s equel we a ssume tha t Ω ≤ 0 . 5 so that the minimal samp ling rate of Theorem 1 is exactly twice the Landau rate. It is e asy to s ee that M ⊂ N Ω for Ω = N B T . The refore, for known s pectral suppo rt, the Landau rate is N B . Despite the fact that M is a true subset of N N B T , the proo f of The orem 1 can be ad opted to show that a minimal density of 2 N B is required s o that stable p erfect rec onstruction is p ossible for sign als from M . W e point out that both Landau’ s a nd Theorem 1 state a lower bound b ut do no t pro v ide a me thod to achieve the bounds. The rest o f the pape r is de voted to developing a reconstruction me thod that approa ches the minimal sampling rate o f Theorem 1. I V . U N I V E R S A L S A M P L I N G This section revie ws multi-coset samp ling which is u sed in our development. W e also briefly explain the fundamentals of kn own-spectrum reconstruction as de ri ved in [8]. A. Multi-cose t sampling Uniform samp ling o f x ( t ) at the Nyqu ist rate results in samples x ( t = nT ) that contain all the information about x ( t ) . Multi-coset s ampling is a selec tion o f certain sample s from this grid. The un iform g rid is divided into blocks of L co nsecutive s amples. A co nstant set C of length p de scribes the indices o f p samples that are kept in each block while the rest are ze roed ou t. The se t C = { c i } p i =1 is re ferred to as the s ampling pattern where 0 ≤ c 1 < c 2 < ... < c p ≤ L − 1 . (13) Define the i th sampling s equence for 1 ≤ i ≤ p as x c i [ n ] = ( x ( t = nT ) n = mL + c i , for some m ∈ Z 0 otherwise. (14) The samp ling s tage is implemen ted by p uniform s ampling sequ ences with period 1 / ( LT ) , where the i th sampling sequen ce is shifted by c i T from the origin. Therefore, a multi-coset system is un iquely cha racterized by the 9 parameters L, p and the sampling pa ttern C . Direct c alculations s how that [8] X c i ( e j 2 π f T ) = 1 LT L − 1 X r =0 exp  j 2 π L c i r  X  f + r LT  , (15) ∀ f ∈ F 0 =  0 , 1 LT  , 1 ≤ i ≤ p, where X c i ( e j 2 π f T ) is the discrete-time Fourier transform (DTFT) of x c i [ n ] . Thu s, the goal is to choos e pa rameters L, p, C such that X ( f ) can be rec overed from (15). For our purposes it is c on venient to expres s (15) in a matrix form a s ( ¯ y )( f ) = ( ¯ A ) ( ¯ x )( f ) , ∀ f ∈ F 0 , (16) where ( ¯ y )( f ) is a vector of leng th p whos e i th elemen t is X c i ( e j 2 π f T ) , a nd the vector ( ¯ x )( f ) con tains L u nknowns for each f ( ¯ x ) i ( f ) = X  f + i LT  , 0 ≤ i ≤ L − 1 , f ∈ F 0 . (17) The matrix ( ¯ A ) depen ds on the parame ters L, p and the set C but not on x ( t ) a nd is defi ned b y ( ¯ A ) ik = 1 LT exp  j 2 π L c i k  . (18) Dealing with rea l-v alue d multi-band signals requires simple mod ifications to (16). The se adjustments are de tailed in Appen dix A. The Beurling lo wer de nsity (i.e. the average s ampling rate) of a multi-coset sa mpling set is 1 T A VG = p LT , (19) which is lower than the Nyq uist rate for p < L . Howev e r , an a verage s ampling rate ab ove the Landau rate is no t sufficient for kn own-spectrum reconstruction. Additional con ditions a re n eeded as explaine d in the next section. B. Kn own-spectru m r ec onstruction a nd univ ersality The p resentation of the recon struction is simplified u sing CS sparsity notation. A vector ( ¯ v ) is called K -spars e if the number of non-zero values in ( ¯ v ) is no greater than K . Using the ℓ 0 pseudo -norm the spa rsity of ( ¯ v ) is expressed a s k ( ¯ v ) k 0 ≤ K . W e use the follo wing defi nition of the K ruskal-rank of a ma trix [14]: Definition 1: The Kruskal-rank of ( ¯ A ) , deno ted as σ ( ( ¯ A )) , is the max imal number q such that every set of q columns of ( ¯ A ) is linearly independent. Observe that for every f ∈ F 0 the s ystem of (16) has less e quations than un knowns. T herefore, a prior on ( ¯ x )( f ) must be used to allo w for recovery . In [8] it is assumed that the information a bout the band locations is av ailable in the re construction s tage. This information supp lies the se t I ( ( ¯ x )( f )) for every f ∈ F 0 . W ithout a ny 10 additional prior the foll owing co ndition is necessa ry for known-spectrum pe rfect reco nstruction ( ¯ x )( f ) is p -sparse , ∀ f ∈ F 0 . (20) Using the Krus kal-rank of ( ¯ A ) a s uf fi cient condition is formulated a s ( ¯ x )( f ) is σ ( ( ¯ A )) -sparse , ∀ f ∈ F 0 . (21) The known-spectrum recon struction of [8] basically restricts the column s of ( ¯ A ) to I ( ( ¯ x )( f )) an d in verts the resulting matrix in order to recover ( ¯ x )( f ) . A sampling pattern C that yields a fully Krus kal-rank ( ¯ A ) is called univ ers al and correspond s to σ ( ( ¯ A )) = p . Therefore, the set of signa ls tha t are consisten t with (21) is the broadest possible if a un i versal sa mpling pattern is u sed. As we show later , choosing L ≤ 1 B T , p ≥ N and a un i versal pattern C makes (21) valid for every signal x ( t ) ∈ M . Finding a uni versal pa ttern C , namely one that results in a fully Kruska l-rank ( ¯ A ) , is a combinatorial p rocess. Several specific constructions of sa mpling patterns that are proved to be univ ersa l are given in [8],[10]. In particular , choosing L to be p rime renders every pattern universal [10]. T o summarize, ch oosing a universal p attern allows recovery o f any x ( t ) sa tisfying (20) when the ban d locations are known in the reco nstruction. W e next cons ider blind s ignal recovery using un i versal sampling pa tterns. V . S P E C T RU M - B L I N D R E C O N S T RU C T I O N In this section we develop the theo ry need ed for SBR. Thes e results are then us ed in the next sec tion to construct two e f fic ient a lgorithms for blind signal reco nstruction. The t h eoretical results are dev oted in the follo wing steps: W e first note that when considering blind-reconstruction, we ca nnot use the prior of [8]. In Section V -A we pres ent a different prior that do es not a ssume the information about the band locations . Us ing this prior we develop a su f fic ient c ondition for blind perfect reco nstruction which is very s imilar to (21). Furthermore, we prov e that under certain conditions on L, p, C , perfec t reconstruction is possible for every signa l in M . W e then prese nt the basic SBR paradigm in Section V -B. T he main res ult of the paper is transforming the c ontinuous system of (16) into a finite d imensional problem without using disc retization. In Section V -C we de velop two propo sitions for this pu rpose, a nd presen t the CTF block. A. Co nditions for blind pe rfect reconstruction Recall that for e very f ∈ F 0 the sy stem of (16) is und etermined sinc e there are fewer equations than unknowns. The prior ass umed in this paper is that for every f ∈ F 0 the vector ( ¯ x )( f ) is sp arse but in contrast to [8] the location of the non-zero values is unknown. Clea rly , in this case (20) is still ne cessa ry for blind pe rfect rec onstruction. The follo wing theorem from the CS literature is used to provide a su f fic ient cond ition. 11 Theorem 2: Su ppose ( ¯ ¯ x ) is a solution of ( ¯ y ) = ( ¯ A ) ( ¯ x ) . If k ( ¯ ¯ x ) k 0 ≤ σ ( ( ¯ A )) / 2 then ( ¯ ¯ x ) is the unique sp arsest solution of the system. Theorem 2 and its proof are given in [11], [15] with a slightly different notation of Spark( A ) ins tead of the Kruskal-rank of ( ¯ A ) . Note that the condition of the theorem is n ot nec essary as there are examples that the sparses t solution ( ¯ ¯ x ) of ( ¯ y ) = ( ¯ A ) ( ¯ x ) is un ique while ( ¯ ¯ x ) > σ ( ( ¯ A )) / 2 . Using Theorem 2 , it is evident that p erfect rec onstruction is poss ible for e very s ignal satisfying ( ¯ x )( f ) is σ ( ( ¯ A )) 2 -sparse , ∀ f ∈ F 0 . ( 2 2) As be fore, cho osing a univ e rsal pattern makes the set of s ignals that conform with (22 ) the widest pos sible. Note that a factor of two distingu ishes betwee n the sufficient c onditions of (21) and of (22), and results from the f ac t that here we do n ot k now the loc ations of the n on-zero values in ( ¯ x )( f ) . Note that (22) provides a condition under which perfect reconstruction is pos sible, ho wever , it is still un clear how to find the original signal. Although the problem is similar to that described in the CS literature, here finding the uniqu e spa rse vector must be solved for e ach value f in the c ontinuous interval F 0 , which clearly canno t be implemented. In practice, conditions (21) and (22) are ha rd to verify since they re quire knowledge of x ( t ) and depe nd o n the p arameters of the multi-coset sampling. W e the refore prefer to develop c onditions on the class M which characterizes multi-band signa ls bas ed on the ir intrinsic properties: the number of bands and the ir widths. It is more likely to know the v alues of N and B in advance than to know if the sign als to be sampled satisfy (21) o r (22). The follo wing the orem des cribes h ow to c hoose the p arameters L, p and C so tha t the s uf fic ient c onditions for perfect recon struction hold true for every x ( t ) ∈ M , na mely it is a unique solution of (16). Th e the orem is valid for both known and blind reco nstruction with a slight dif ference res ulting fr o m the factor of two in the sufficient con ditions. Theorem 3 (Uniquenes s): Let x ( t ) ∈ M be a multi-band s ignal. If: 1) The value of L is limited by L ≤ 1 B T , (23) 2) p ≥ N for known reconstruction or p ≥ 2 N for blind, 3) C is a u ni versa l pattern, then, for e very f ∈ F 0 , the vector ( ¯ x )( f ) is the uniqu e solution of (16). Pr oof: If L is limited by (23) then for the i th band T i = [ a i , b i ] we have λ ( T i ) ≤ B ≤ 1 LT , 1 ≤ i ≤ N . Therefore, f ∈ T i implies f + k LT / ∈ T i , ∀ k 6 = 0 . 12 According to (17) for ev ery f ∈ F 0 the vector ( ¯ x )( f ) tak e s the v a lues of X ( f ) on a set of L points spaced by 1 /LT . Conse quently , the numbe r of non-zero values in ( ¯ x )( f ) is no g reater tha n the number of the b ands, na mely ( ¯ x )( f ) is N -sparse. Since C is a u ni versa l pattern, σ ( ( ¯ A )) = p . This implies that c onditions (21) an d (22 ) are satisfied . Note that the c ondition on the value of p implies the minimal sampling rate req uirement. T o see this, s ubstitute (23) into (19): 1 T A VG = p LT ≥ pB . (24) As po inted ou t in the end of Section III-B, if the signals are kn own to lie in M then the La ndau rate is N B , which is implied b y p ≥ N . Theorem 1 requires an average sampling rate of 2 N B , which can be gua ranteed if p ≥ 2 N . B. Re construc tion paradigm The g oal of our reconstruction s cheme is to recover the signal x ( t ) from the set of sequ ences x c i [ n ] , 1 ≤ i ≤ p . Equiv alently , the a im is to recon struct ( ¯ x )( f ) of (16 ) for every f ∈ F 0 from the inpu t data ( ¯ y )( f ) . A straight forward ap proach is to find the s parsest solution ( ¯ x )( f ) on a dense grid of f ∈ F 0 . Howev e r , this discretization strategy c annot guaran tee perfect rec onstruction. In contrast, our a pproach is exact and do es not require disc retization. Our recons truction p aradigm is targeted at finding the diversit y set w hich depen ds on x ( t ) and is defined as S = [ f ∈F 0 I ( ( ¯ x )( f )) . (25) The SBR algorithms we dev elop in Section VI are aimed at recovering the set S . W ith the knowledge of S perfect reconstruction of ( ¯ x )( f ) is possible for e very f ∈ F 0 by noting tha t (16) ca n b e written as ( ¯ y )( f ) = ( ¯ A ) S ( ¯ x ) S ( f ) . (26) If the diversity set of x ( t ) s atisfies | S | ≤ σ ( ( ¯ A )) , (27) then ( ( ¯ A ) S ) † ( ¯ A ) S = I , (28) where ( ¯ A ) S is of s ize p × | S | . Multiplying bo th sides of (26) by ( ( ¯ A ) S ) † results in: ( ¯ x ) S ( f )= ( ( ¯ A ) S ) † ( ¯ y )( f ) , ∀ f ∈ F 0 . (29) From (25), ( ¯ x ) i ( f ) = 0 , ∀ f ∈ F 0 , i / ∈ S. (30) 13 Thus, once S is known, and as long a s (27) holds , p erfect rec onstruction can be obtained by (29 )-(30). As we shall se e later on (27) is implied b y the condition required to transform the problem into a finite dimensional on e. Furthermore, the following propos ition shows that for x ( t ) ∈ M , (27) is implied by the p arameter selection of T heorem 3 . Pr opos ition 1: If L is limited by (23) then | S | ≤ 2 N . If in addition p ≥ 2 N and C is un i versal then for every x ( t ) ∈ M , the set S satisfie s (27). Pr oof: T he bands are continuous interv als uppe r bounded by B . From (17) it follows that ( ¯ x )( f ) is co nstructed by dividing F into L equal intervals o f length 1 /LT . Therefore if L is limited b y (23) the n each b and can either be fully c ontained in one o f the se intervals or it ca n be split betwe en two c onsecu ti ve intervals. Sinc e the nu mber of ba nds is no more than N it follows that | S | ≤ 2 N . W ith the additional conditions we h av e that σ ( ( ¯ A )) = p ≥ 2 N ≥ | S | . As we des cribed, our gen eral s trategy is to determine the diversity s et S and then rec over x ( t ) via (29 )-(30). In the non-blind se tting, S is k nown, and therefore if it satisfie s (27) the n the same equa tions can be used to recover x ( t ) . Howe ver , note that when the ban d locations are known, we may use a value of p that is smaller than 2 N since the sa mpling rate can be reduc ed. Therefore, (27) ma y not hold. Nonetheless , it is shown in [8], tha t the frequency axis can be divided into intervals such tha t this approa ch c an be used over ea ch frequency interv a l. Therefore, onc e the set S is recov e red there is no es sential difference between known and blind recon struction. C. F ormulation of a finite d imensional p r oblem The set of eq uations of (16) cons ists o f an infinite number of linear systems bec ause of the continuou s variable f . Furthermore, the expression for the d i versity set S giv en in (25) in volves a un ion over the same c ontinuous variable. The main result of this pap er is that S c an be rec overed exactly using only on e finite dimension al problem. In this s ection we develop the unde rlying the oretical res ults that a re u sed for this purpose . Consider a gi ven T ⊆ F 0 . Multiplying each side of (16) by its conjugate transpo se we hav e ( ¯ y )( f ) ( ¯ y ) H ( f ) = ( ¯ A ) ( ¯ x )( f ) ( ¯ x ) H ( f ) ( ¯ A ) H , ∀ f ∈ T . (31) Integrating both sides over the continuous variable f giv es ( ¯ Q ) = ( ¯ A ) ( ¯ Z ) 0 ( ¯ A ) H , (32) with the p × p matrix ( ¯ Q ) = Z f ∈T ( ¯ y )( f ) ( ¯ y ) H ( f ) d f  0 , (33) and the L × L matrix ( ¯ Z ) 0 = Z f ∈T ( ¯ x )( f ) ( ¯ x ) H ( f ) d f  0 . (34) 14 Define the diversity set of the interval T as S T = [ f ∈T I ( ( ¯ x )( f )) . (35) Now , ( ( ¯ Z ) 0 ) ii = Z f ∈T | ( ¯ x ) i ( f ) | 2 d f . This means tha t ( ( ¯ Z ) 0 ) ii = 0 if and only if ( ¯ x ) i ( f ) = 0 , ∀ f ∈ T , which implies that S T = I ( ( ¯ Z ) 0 ) . The next propos ition is use d to determine wh ether ( ¯ Z ) 0 can b e found by a finite d imensional problem. T he proposition is s tated for gene ral matrices ( ¯ Q ) , ( ¯ A ) . Pr opos ition 2: Suppos e ( ¯ Q )  0 of s ize p × p a nd ( ¯ A ) are given matri c es. L et ( ¯ Z ) be any L × L matrix s atisfying ( ¯ Q ) = ( ¯ A ) ( ¯ Z ) ( ¯ A ) H , (36a) ( ¯ Z )  0 , (36b) | I ( ( ¯ Z )) | ≤ σ ( ( ¯ A )) . (36c) Then, rank( ( ¯ Z )) = rank ( ( ¯ Q )) . If, in addition, | I ( ( ¯ Z )) | ≤ σ ( ( ¯ A )) 2 , (36d) then, ( ¯ Z ) is the unique so lution of (36a)-(36d). Pr oof: Let ( ¯ Z ) satisfy (36a)-(36c). Define r Q = rank( ( ¯ Q )) , r Z = rank( ( ¯ Z )) . Sinc e ( ¯ Z )  0 it can be decompo sed as ( ¯ Z ) = ( ¯ P ) ( ¯ P ) H with ( ¯ P ) of size L × r Z having orthogo nal columns. From (36a), ( ¯ Q ) = ( ( ¯ A ) ( ¯ P ))( ( ¯ A ) ( ¯ P )) H . (37) It can be eas ily be conclude d that I ( ( ¯ Z )) = I ( ( ¯ P )) , and thus | I ( ( ¯ P )) | ≤ σ ( ( ¯ A )) . The follo wing lemma whose proof is g i ven in App endix B ensures that the matrix ( ¯ A ) ( ¯ P ) of size p × r Z also has full c olumn rank. Lemma 1: For ev e ry two ma trices ( ¯ A ) , ( ¯ P ) , if | I ( ( ¯ P )) | ≤ σ ( ( ¯ A )) then r ank( ( ¯ P )) = r ank( ( ¯ A ) ( ¯ P )) . Since for e very ma trix ( ¯ M ) it is true that rank( ( ¯ M )) = rank( ( ¯ M ) ( ¯ M ) H ) , (37) implies r Z = r Q . For the secon d part of Propo sition 2 suppose tha t ( ¯ Z ) , ( ¯ ˜ Z ) both satisfy (36a),(36b),(36d) . From the first part, rank( ( ¯ Z )) = rank ( ( ¯ ˜ Z )) = r Q . Follo wing the e arlier dec ompositions we write ( ¯ Z ) = ( ¯ P ) ( ¯ P ) H , I ( ( ¯ Z )) = I ( ( ¯ P )) (38) ( ¯ ˜ Z ) = ( ¯ ˜ P ) ( ¯ ˜ P ) H , I ( ( ¯ ˜ Z )) = I ( ( ¯ ˜ P )) . 15 In addition, | I ( ( ¯ P )) | ≤ σ ( ( ¯ A )) 2 , | I ( ( ¯ ˜ P )) | ≤ σ ( ( ¯ A )) 2 . (39) From (36a), ( ¯ Q ) = ( ( ¯ A ) ( ¯ P ))( ( ¯ A ) ( ¯ P )) H = ( ( ¯ A ) ( ¯ ˜ P ))( ( ¯ A ) ( ¯ ˜ P )) H , (40) which implies that ( ¯ A )( ( ¯ P ) − ( ¯ ˜ P ) ( ¯ R )) = 0 , (41) for some u nitary matrix ( ¯ R ) . It is easy to see that (39) res ults in | I ( ( ¯ ˜ P ) ( ¯ R )) | ≤ σ ( ( ¯ A )) / 2 . Therefore the matri x ( ¯ P ) − ( ¯ ˜ P ) ( ¯ R ) has a t mos t σ ( ( ¯ A )) non-identical ze ro rows. Applying Lemma 1 to (41) results in ( ¯ P ) = ( ¯ ˜ P ) ( ¯ R ) . Substituting this into (38 ) we have that ( ¯ Z ) = ( ¯ ˜ Z ) . The following proposition sh ows how to cons truct the matrix ( ¯ Z ) by finding the sparses t solution of a linear system. Pr opos ition 3: Consider the setting o f Proposition 2 and ass ume ( ¯ Z ) satisfie s (36d). Let r = rank( ( ¯ Q )) and define a matrix ( ¯ V ) of size p × r u sing the d ecompos ition ( ¯ Q ) = ( ¯ V ) ( ¯ V ) H , s uch that ( ¯ V ) has r orthogonal columns. Then the linea r system ( ¯ V ) = ( ¯ A ) ( ¯ U ) (42) has a u nique s parsest solution matrix ( ¯ U ) 0 . N amely , ( ¯ V ) = ( ¯ A ) ( ¯ U ) 0 and | I ( ( ¯ U ) 0 ) | is minimal. Moreov er , ( ¯ Z ) = ( ¯ U ) 0 ( ¯ U ) H 0 . Pr oof: S ubstitute the decomp osition ( ¯ Q ) = ( ¯ V ) ( ¯ V ) H into (36a) and let ( ¯ Z ) = ( ¯ P ) ( ¯ P ) H . Th e res ult is ( ¯ V ) = ( ¯ A ) ( ¯ P ) ( ¯ R ) for so me unitary ( ¯ R ) . Th erefore, the linea r system of (42) has a solution ( ¯ U ) 0 = ( ¯ P ) ( ¯ R ) . It is easy to see that I ( ( ¯ U ) 0 ) = I ( ( ¯ P )) = I ( ( ¯ Z )) , thus (36d) res ults in | I ( ( ¯ U ) 0 ) | ≤ σ ( ( ¯ A )) / 2 . Applying Theorem 2 to each of the columns of ( ¯ U ) 0 provides the u niqueness of ( ¯ U ) 0 . It is trivial that ( ¯ Z ) = ( ¯ U ) 0 ( ¯ U ) H 0 . Using the same arguments as in the proof it is ea sy to c onclude that I ( ( ¯ Z )) = I ( ( ¯ U ) 0 ) , s o that S T can be found directly from the so lution matrix ( ¯ U ) 0 . In particular , we develop the Continuous to F inite (CTF) block wh ich determines the diversit y set S T of a gi ven frequency interval T . Fig. 2 presents the CTF b lock that contains the flow of transforming the continuous linear system of (16) on the interval T into the finite dimens ional problem of (42) and the n to the recovery of S T . The role o f Propo sitions 2 an d 3 is also illustrated. The CTF block is the heart of the SBR scheme which we discu ss next. In the CS literature, the linear system of (42) is referred to as an MMV system. Theoretical res ults regarding the sparsest solution matrix of an MMV system are given in [15 ]. Finding the solution matrix ( ¯ U ) 0 is k nown to be NP-hard [12]. Several sub -optimal e f fic ient algorithms for finding ( ¯ U ) 0 are gi ven in [16]. Some of the m can indicate a s ucces s recovery of ( ¯ U ) 0 . W e explain which class o f algorithms has this property in Section VI-A. 16 Fig. 2. Continuous to finite block (CTF). This block determines the div ersity set S T of a given interval T . V I . S B R A L G O R I T H M S The theoretical results developed in the pre viou s se ction are now use d in order to construct the di versity set S which enab les the recovery of x ( t ) via (29)-(30). W e begin b y defining a clas s A o f signals. T he SBR4 algorithm is then pres ented a nd is p roved to guarantee perfect reco nstruction for signa ls in A . W e then show that in orde r to ens ure that M ⊆ A the sa mpling rate must be a t lea st 4 N B , which is twice the minimal rate stated in Theorem 1. T o improve on this result, we de fine a class B of signa ls, and introduce a concep tual method to perfectly reconstruct this class. The SBR2 a lgorithm is developed s o tha t it e nsures exact recovery for a subset of B . W e then prov e that M is c ontained in this subset ev e n for sampling at the minimal rate. Howe ver , the comp utational complexity o f SBR2 is higher than tha t of SBR4. S ince universal pa tterns lead to the largest sets A a nd B , we a ssume throughou t this se ction that univ e rsal patterns are used, which res ults in σ ( ( ¯ A )) = p . A. Th e SBR4 algor ithm Define the c lass A K of signals A K = { sup p X ( f ) ⊆ F and | S | ≤ K } , (43) with S giv e n b y (25). Let T = F 0 , a nd observe that a multi-coset system with p ≥ 2 K ensures tha t all the conditions of Propos ition 2 a re valid for every x ( t ) ∈ A K . Thus , applying the CT F block on T = F 0 results in a unique sparsest solution ( ¯ U ) 0 , with S = I ( ( ¯ U ) 0 ) . Th e reco nstruction of the signal is then carried o ut by (29)-(30). W e note that (27) is valid a s it repres ents the c lass A p that conta ins A K for p ≥ 2 K . Algorithm 1, n amed SBR4, follo ws the steps of the CTF block in Fig. 2 to recover the d i versity set S from ( ¯ y )( f ) , for any x ( t ) ∈ A K . The a lgorithm also outputs a n indication flag which we discuss later on . The SBR4 algorithm gu arantees pe rfect reco nstruction of sign als in A K from samples at twice the Land au rate, which is also the lo wer bound sta ted in Theorem 1. T o see this, ob serve that (25 ) impli e s that every x ( t ) ∈ A K must sa tisfy λ (supp X ( f )) ≤ K LT . (44) 17 Algorithm 1 SBR4 Input: ( ¯ y )( f ) , Assume: σ ( ( ¯ A )) = p Output: the set S , flag 1: Set T = F 0 2: Compute the matri x ( ¯ Q ) b y (33 ) 3: Decompos e ( ¯ Q ) = ( ¯ V ) ( ¯ V ) H according to Prop osition 3 4: Solve the MMV s ystem ( ¯ V ) = ( ¯ A ) ( ¯ U ) for the spa rsest s olution ( ¯ U ) 0 5: S = I ( ( ¯ U ) 0 ) 6: flag = {| S | ≤ p 2 } 7: return S , flag Although A K is n ot a subspac e, we use (44) to s ay that the Lan dau rate for A K is K /LT a s it contains subs paces whose widest support is K/LT . As we p roved, p ≥ 2 K en sures p erfect reconstruction for A K . Substituting the smallest poss ible value p = 2 K into (19) resu lts in an average s ampling rate of 2 K/LT . It is easy to se e that flag is equal to 1 for every signal in A K . Howev er , wh en a sub -optimal algorithm is use d to s olve the MMV in s tep 4 we c annot gua rantee a c orrect s olution ( ¯ U ) 0 . Thus, flag= 0 indicates that the particular MMV method we used failed, a nd we may try a different MMV a pproach. Existing a lgorithms for MMV s ystems ca n be classified into two groups . The first group c ontains algo rithms that se ek the sparsest solution matrix ( ¯ U ) 0 , e.g . Basis Purs uit [17] or Matching Pursuit [18 ] with a termination criterion based on the res idual. The other contains metho ds that approximate a s parse s olution a ccording to user specifica tion, e.g. Ma tching Purs uit with a predetermined number of iterations. Using a technique from the latter group n eutralizes the indication flag as the ap proximation is always s parse. Therefore, this set of algorithms should be av oide d if an ind ication is des ired. An important advantage of algorithm SBR4 is that the matrix ( ¯ Q ) can be computed in the time do main fr o m the known s equenc es x c i [ n ] , 1 ≤ i ≤ p . The computation in volves a set of digital filters that do not de pend on the signa l a nd thus can be designed in adv a nce. The exa ct details are given in Appe ndix C. The d rawback of the set A K , is that typically we do not kn ow the value of K . Moreover , ev e n if K is known, then usually we do not kn ow in advance whether x ( t ) ∈ A K as A K does not c haracterize the signa ls acco rding to the number of ban ds a nd their widths . Therefore, we would like to determine conditions that ensure M ⊆ A K . Proposition 1 s hows that for x ( t ) ∈ M the set S satisfies | S | ≤ 2 N if L ≤ 1 /B T . T hus, under this c ondition on L w e have M ⊆ A 2 N , which in turn implies p = 4 N as a minimal v alue for p . Co nseque ntly , SBR4 guaran tees perfect reco nstruction for M under the re strictions L ≤ 1 /B T and p ≥ 4 N . Howe ver , the Landau rate for M is N B , while p = 4 N implies a minimal sampling rate of 4 N B . Indeed, substituting p = 4 N and L ≤ 1 /B T into (19) we ha ve p LT ≥ 4 N T 1 B T = 4 N B . (45) In con trast, it follo ws from Theo rem 3 that p ≥ 2 N is sufficient for uniqueness of the so lution. The reaso n for 18 the factor of two in the sampling rate is that ( ¯ x )( f ) is N -sp arse for eac h s pecific f ; ho wever , whe n combining the frequencies , the maximal s ize of S is 2 N . T he SBR2 algorithm, developed in the next section, capitalizes on this difference to regain the factor of two in the sampling rate, and thus achieves the minimal rate, at the expens e of a more complicated reconstruction method . B. Th e SBR2 algor ithm W e now would like to reduce the s ampling rate requ ired for signals of M to its minimum, i.e . twice the Landa u rate. T o this end, we introduce a se t B K for which SBR2 guarantees p erfect reconstruction, and then p rove that M ⊆ B N if p ≥ 2 N . Consider a partiti on of F 0 into M con secutive intervals d efined by 0 = ¯ d 1 < ¯ d 2 < · · · < ¯ d M +1 = 1 LT . For a gi ven p artition set ¯ D = { ¯ d i } we d efine the se t of signals B K, ¯ D = { sup p X ( f ) ⊆ F a nd | S [ ¯ d i , ¯ d i +1 ] | ≤ K, 1 ≤ i ≤ M } . Clearly , if p ≥ 2 K then we can pe rfectly reconstruct e very x ( t ) ∈ B K, ¯ D by applying the CTF block to eac h of the intervals [ ¯ d i , ¯ d i +1 ] . W e now define the s et B K as B K = [ ¯ D B K, ¯ D , ( 46 ) which is the union o f B K, ¯ D over all choices of partition sets ¯ D and integers M . Note that neither B K nor B K, ¯ D is a subs pace. If we are able to find a pa rtition ¯ D such that x ( t ) ∈ B K, ¯ D , then x ( t ) ca n be perfectly reconstructed using p ≥ 2 K . Since the Land au rate for B K is K/LT , this approach requires the minimal sampling rate 2 . The following propos ition shows tha t if the p arameters are chose n properly , then M ⊆ B N . Thus , p ≥ 2 N a nd a method to fi nd ¯ D of x ( t ) is sufficient for perfect recon struction of x ( t ) ∈ M . Pr opos ition 4: If L, p, C are s elected according to Theorem 3 then M ⊆ B N . Pr oof: In the proof of Theorem 3 we showed that under the conditions of the theorem, ( ¯ x )( f ) is N -sparse for every f ∈ F 0 . The p roof of the proposition then follows from the following lemma [8]: Lemma 2: If x ( t ) is a mu lti-band signal with N bands sa mpled by a multi-coset sys tem then there exists a partition set ¯ D = { ¯ d i } with M = 2 N + 1 interv als suc h that I ( ( ¯ x )( f )) is a constant s et over the interval [ ¯ d i , ¯ d i +1 ] for 1 ≤ i ≤ M . Lemma 2 impli e s that | S [ ¯ d i , ¯ d i +1 ] | ≤ N for every 1 ≤ i ≤ M = 2 N + 1 wh ich me ans that x ( t ) ∈ B N , ¯ D . So f ar we sho we d that M ⊆ B N , h owe ver to recover x ( t ) we nee d a method to find ¯ D in practice; Lemma 2 2 under t he con vention discussed for A K . 19 only ensu res its existenc e. Given the da ta ( ¯ y )( f ) , our strategy is a imed at find ing any partition se t D such that ˆ S = | D |− 1 [ i =0 S [ d i ,d i +1 ] (47) is e qual to S , and su ch that | S [ d i ,d i +1 ] | ≤ K for every 1 ≤ i ≤ M . As long as (27) holds, once we fin d S the solution is exactly recovered via (29)-(30). T o fin d S , we ap ply the CTF b lock on each interv al [ d i , d i +1 ] . If p ≥ 2 K , then the cond itions o f Propo sition 2 a re valid, a u nique s olution is guaranteed for each interval. Since for p = 2 K (27) is valid for A 2 K , our method gua rantees perfect rec onstruction o f signa ls in B K ∩ A 2 K . As always, using a universal pattern ma kes the set of s ignals B K ∩ A 2 K the largest. S ince the Land au rate for B K ∩ A 2 K is K/LT this ap proach allows for the minimal sampling rate whe n p = 2 K . In order to fi nd D we su ggest a bi-section process on F 0 . W e initialize T = F 0 and s eek S T . If S T does not satisfy some con dition explaine d be lo w , then we ha lve T into T 1 and T 2 and determine S T 1 and S T 2 . The bi-section proces s is repe ated several times until the conditions are met, or until it rea ches an interval width of no more than ǫ . The set ˆ S is then de termined according to (47). W e now describe the conditions for which a g i ven T ⊆ F 0 is halved. The matrix ( ¯ Z ) 0 of (34) satisfies the constraints (36a)-(36b). Since x ( t ) ∈ A 2 K and p ≥ 2 K (36c ) is a lso valid. Howe ver , the last constraint (36d ) of Proposition 2 is not guarantee d as it requires a stronger condition | S T | ≤ K = p/ 2 . Note that this co ndition is satisfied immediately if D = ¯ D sinc e x ( t ) ∈ B K . W e sug gest to approximate the value S T = | I ( ( ¯ Z ) 0 ) | by rank( ( ¯ Q )) , a nd solve the MMV s ystem for the sparsest so lution on ly if rank( ( ¯ Q )) ≤ p/ 2 . This a pproximation is moti vated by the f ac t that for any ( ¯ Z )  0 it is true that r an k ( ( ¯ Z )) ≤ | I ( ( ¯ Z )) | . F rom Proposition 2 we have that rank( ( ¯ Z ) 0 ) = rank( ( ¯ Q )) which results in rank( ( ¯ Q )) ≤ | I ( ( ¯ Z )) | . (48) Howe ver , only specia l multi-band signals res ult in strict ineq uality in (48). Therefore, a n interval T that produce s rank( ( ¯ Q )) > p/ 2 is ha lved. Othe rwise, we apply the CTF block for this T as suming that (48) ho lds with equality . As in SBR4 the flag indica tes a correc t solution for x ( t ) ∈ B K ∩ A 2 K . The refore, if the flag is 0 we halve T . These recons truction steps are d etailed in Algorithm 2, named SBR2 . It is important to no te tha t SVR2 is sub -optimal, since the final output of the algorithm ˆ S may not be equa l to S e ven for x ( t ) ∈ B K ∩ A 2 K . One reaso n this can happe n is if strict inequa lity h olds in (48) for some interval T . In this scena rio step 7 is ex e cuted even thou gh ( ¯ Z ) 0 does no t sa tisfy (36d). For example, a signal x ( t ) with tw o equal width b ands [ a 1 , a 1 + W ] an d [ a 2 , a 2 + W ] such that j a 1 LT k = j a 2 LT k = γ (49) and γ + W ∈ F 0 . If x ( t ) also satisfies X ( f − a 1 ) = X ( f − a 2 ) , ∀ f ∈ [0 , W ] , (50) 20 Algorithm 2 SBR2 Input: T , Initiali z e: T = F 0 , Assu me: σ ( ( ¯ A )) = p Output: a set ˆ S 1: if λ ( T ) ≤ ǫ then 2: return ˆ S = {} 3: end if 4: Compute the matri x ( ¯ Q ) b y (33 ) 5: if rank( ( ¯ Q )) ≤ p 2 then 6: Decompos e ( ¯ Q ) = ( ¯ V ) ( ¯ V ) H 7: Solve MMV sy stem ( ¯ V ) = ( ¯ A ) ( ¯ U ) 8: ˆ S = I ( ( ¯ U ) 0 ) 9: else 10: ˆ S = {} 11: e nd if 12: if (rank( ( ¯ Q )) > p 2 ) or ( | ˆ S | > p 2 ) then 13: split T into two equal width intervals T 1 , T 2 14: ˆ S (1) = SBR2 ( T 1 ) 15: ˆ S (2) = SBR2 ( T 2 ) 16: ˆ S = ˆ S (1) ∪ ˆ S (2) 17: e nd if 18: return ˆ S then it can be verified that | I ( ( ¯ Z ) 0 ) | = 2 while rank( ( ¯ Z ) 0 ) = rank( ( ¯ Q )) = 1 o n the interval T = [ γ , γ + W ] . This is of course a rare special case. Another re ason is a s ignal for which the algorithm rea ched the termination step 1 for so me sma ll enou gh interval. This scena rio can ha ppen if two or more points of ¯ D reside in an interval width o f ǫ . As a n empty s et ˆ S is returned for this interval, the fin al output may be miss ing some of the eleme nts of S . Clearly , the value of ǫ influ ences the amount o f case s of this type. W e note that s ince we do not rely on D = ¯ D the miss ing values are typ ically recovered from other intervals. Thus , b oth o f these sources o f error are very un common. The mo st c ommon case in which S BR2 ca n fail is due to the use of s ub-optimal algorithms to find ( ¯ U ) 0 ; this issue also occurs in SBR4. As explained b efore, we assume that flag=0 me ans an incorrect solution and halves the interval T . An interesting b ehavior o f MMV method s is that ev en if ( ¯ U ) 0 cannot be found for T , the algorithm may still find a sparse s olution for e ach of its subs ections. Thu s, the indication fla g is also a way to p artially overcome the practical limi ta tions of MMV techniques. Note that the indication property is crucial for SBR2 a s it helps to refine the partiti o n D and re duce the su b-optimality resulting from the MMV algorithm. W e point out that P roposition 4 shows that M ⊆ B N . W e also have that M ⊆ A 2 N from P roposition 1 , which moti vates o ur approach . Th e SBR2 algorithm it s elf does not impose any additional limit a tions on L, p, C other than those of Theo rem 3 required to e nsure the uniqueness of the solution. Th erefore, theoretically , perfect reconstruction for M is gu aranteed if the samples are ac quired at the minimal rate, with the exce ption of the special ca ses discu ssed be fore. 21 T ABLE II S P E C T R U M - B L I N D R E C O N S T R U C T I O N M E T H O D S F O R M U LT I - B A N D S I G N A L S WKS theorem SBR4 SBR2 Sampling method Uniform Multi-coset Multi-coset Fully-blind Y e s Y e s Y e s # Uniform se quence s 1 p p Minimal sa mpling rate Nyquist 2 × La ndau 2 × La ndau Achieves lo wer bound of T heorem 1 No Y e s Y e s Recons truction method Ideal low pa ss SBR4 SBR2 T ime complexity constant 1 MMV s ystem bi-section + fi nite # of MMV Applicability supp X ( f ) ⊆ F x ( t ) ∈ A K x ( t ) ∈ B K ∩ A 2 K 3 Indication No f o r x ( t ) ∈ A K only No The c omplexity of SBR2 is dictated by the number of iterations of the bi-section proce ss, which is also af fec ted by the b ehavior of the MMV algorithm that is used. Numerical experiments in Section VII show that e mpirically SBR2 con verges su f fic iently fast for practical usa ge. Finally , we e mphasize that SBR2 does not provide an indica tion on the succe ss recovery of x ( t ) even for x ( t ) ∈ M since there is n o way to k now in advance if x ( t ) is a signal of the s pecial type that SBR 2 cann ot recover . C. Com parison between SBR4 a nd SBR2 T able II c ompares the properties of SBR4 and SBR2. W e added the WKS theorem as it also of fers spectrum- blind rec onstruction. Both SBR4 a nd S BR2 a lgorithms recover the set S acc ording to the pa radigm stated in Section V -B. Observe that an indication property is available only for S BR4 and only if the signa ls are known to lie in A K . Although both SB R4 and SBR2 can operate at the minimal sampling rate, SBR2 g uarantees perfect reconstruction for a wider se t of signa ls as A K is a tr u e sub set of B K ∩ A 2 K . Considering s ignals from M we have to re strict the parameter se lection. The specific be havior of SBR4 and SBR2 for this scenario is compared in T ab le III. In particular , SBR4 requ ires twice the minimal rate. In the tables, p erfect reco nstruction refers to recon struction with a brute-force MMV method that finds the correct solution. In practice, su b-optimal MMV algorithms may result in failure of rec overy even whe n the other requirements are met. The indic ation fl ag is intend ed to disc over thes e cases . The entire recon struction sch eme is presented in Fig. 3. T he sc heme together with the tables allow for a wise decision on the particular implemen tation o f the system. Clearly , for Ω > 0 . 5 it s hould be preferred to samp le a t the Nyquist rate and to rec onstruct with an ideal low pass filter . For Ω ≤ 0 . 5 we have to choose between SBR4 and SBR2 ac cording to our prior on the signal. T ypica lly , it is natural to a ssume x ( t ) ∈ M for so me v a lues of 3 excep t for special signals discussed in Section VI-B. 22 T ABLE III C O M PA R I S O N O F S B R 4 A N D S B R 2 F O R S I G N A L S I N M SBR4 SBR2 # Uniform se quence s p ≥ 4 N p ≥ 2 N Minimal rate 4 × La ndau 2 × La ndau Lower bound of T h. 1 No Y e s Parameter s election Theorem 3, p ≥ 4 N Theorem 3 Perfect recon struction Y e s Y e s 3 Indication Y e s No Fig. 3. Spectrum-blind reconstruction scheme. N and B and deriv e the required pa rameter s election a ccording to T ab le III. It is obvious that if p ≥ 4 N is use d then SBR4 sho uld be preferred s ince it is less comp licated than S BR2. The trade-off presen ted here between complexity a nd sampling rate also e x ists in the kno wn -spectrum recons truc- tion o f [8]. Sampling a t the minimal rate of Landau requires a recon struction that consists of piecewise c onstant filters. The number of piec es a nd the rec onstruction c omplexity grow with L . This complexity ca n be pre vented by doubling the value of p which also doubles the average sa mpling rate a ccording to (19). Then, (29)-(30) are used to reco nstruct the signa l by only one in version o f a kn own matrix [6]. V I I . N U M E R I C A L E X P E R I M E N T S W e now provide several experiments demons trating the reco nstruction u sing algo rithms SBR4 a nd SBR2 for signals from M . W e also p rovide an example in which the sign als do not lie in the c lass M but in the larger s et implied by A K for SBR4 a nd by B K ∩ A 2 K for SBR2 . 23 A. Se tup The setup d escribed hereafter is u sed as a b asis for all the experiments. Consider an example of the c lass M with F = [0 , 20 GHz ] , N = 4 a nd B = 100 MHz . In order to test the algorithms 100 0 test cases from this class were gen erated rand omly according to the follo wing steps : 1) draw { a i } N i =1 uniformly at rand om from [0 , 20 GHz − B ] . 2) set b i = a i + B for 1 ≤ i ≤ N , and e nsure that the bands do n ot overlap. 3) Generate X ( f ) by X ( f ) =    α ( f ) ( S R ( f ) + j S I ( f )) , f ∈ N S i =1 [ a i , b i ] 0 , otherwise. For every f the values of S R ( f ) and S I ( f ) are drawn indepe ndently from a no rmal distribution with zero mean and unit v a riance. Th e func tion α ( f ) is c onstant in eac h ba nd, a nd is chose n su ch that the b and energy is equ al to e i where e i is selec ted uniformly from [1,5]. The La ndau rate for e ach of the signals is N B = 400 MHz, and thu s the minimal rate req uirement for b lind reconstruction is 800 MHz du e to Theo rem 1. Several mu lti-coset syste ms are con sidered with the follo wing parameters. The value L is common in all the systems. The value o f p is varied from p = N = 4 to p = 8 N = 32 representing 29 dif ferent sys tems. A universal pattern C is con structed by cho osing prime L , since acc ording to [10] this ensure s that ev e ry samp ling pattern is univ e rsal. An expe riment is co nducted by sampling the signals using e ach of the multi-coset sys tems. Each o f these combinations is used as an input to both SBR4 and SBR2 algorithms. W e selected the Multi-Or tho gonal Matching Pursuit (M-OMP) me thod [16] to solve the MMV systems for the spa rsest solution. The empirical suc cess rate of each algorithm is calculated as the ratio of simulations in which the recovered set S is co rrect. B. Sa mpling rate a nd practical limitations W e begin by s electing the lar ges t poss ible value of p rime L sa tisfying (23): L = 199 ≤ 1 B T = 200 . (51) Thus, the minimal rate requiremen t h olds only for p ≥ 2 N . S pecifically , for p = 2 N the sampling ra te is p/LT = 804 MHz. Obs erve that a no n-prime L = 200 would g i ve the minimal rate exactly . This setting is discuss ed later on. Fig. 4 d epicts the emp irical success rate with L = 199 , N = 4 as a func tion of p . It is evident that for p < 2 N the set S c ould no t be recovered by neither of the algorithms since the s ampling rate is below the bou nd of Theorem 1 . As expe cted, SB R2 outperforms SBR4 as it ac hiev e s the same empirical suc cess rate for a lower average sa mpling rate. It is also s een that for p = 4 N the sampling rate is slightly more tha n fou r times the Landau rate. Ind eed, 24 5 10 15 20 25 30 0 0.2 0.4 0.6 0.8 1 p Empirical success rate SBR4 SBR2 Fig. 4. Performance of S BR algorithms wit h L = 199 . 5 10 15 20 0 0.2 0.4 0.6 0.8 1 p Empirical success rate SBR4 SBR2 Fig. 5. Performance of S BR algorithms wit h L = 23 . algorithm SBR4 maintains a high recovery rate for this value of p . The usage of SBR2 with M-OMP maintains a high recovery rate for p/ N = 2 . 6 , which is more than the minimal rate. Other MMV algorithms may be us ed to improve this resu lt, howev e r we use d only M-OMP as it is simple and fast. W e next con sider a sce nario with L = 23 , which clearly s atisfies (23). H ere, for p = N = 4 we have a sampling rate of 3.4 GHz whic h is much h igher than the minimal requirement. T his selection o f L represe nts a practical desire to satisfy the minimal rate requirement with a reduc ed value of p , s ince rea lizing the mu lti-coset s ampling requires p ana log-to-digital devices. Fig. 5 presents the empirical recovery ra te in this case. Note that T able III shows that in order to guaran tee p erfect recons truction for M we need p ≥ 4 N for SBR4, a nd p ≥ 2 N for SBR2. However , these conditions are on ly sufficient. Ind eed, it is evident from Fig. 5 that both algorithms re ach a sa tisfactory recovery rate for lo wer values o f p . In T a ble IV, we tabulate the av erag e run time of one case out of the 100 0 tested. Our experiments were conduc ted on an o rdinary PC de sktop with an Intel CPU running at 2.4GHz and 512MB memory RAM. W e u sed Matlab version 7 to encod e an d execute the algorithms. Note that for L = 199 , p = 2 N we e ncountered a significa nt increase in SBR2 runtime. The reason is that the average sa mpling rate is very close to the minimal possible, thus the rec ursion depth of the algorithm grows as it is harde r to find a suitable partition se t D . For p = 4 N the 25 T ABLE IV A V E R A G E R U N T I M E O F S B R 4 A N D S B R 2 W I T H MO M P ( M S E C ) L = 199 L = 23 SBR4 SBR2 SBR4 SBR2 p = N 7 608 4.2 51.4 p = 2 N 16.1 1034 5.7 6.4 p = 4 N 21.4 24.8 6.7 6.7 runtime d ramatically improves, ho wever in this case SBR4 may be preferred due to the advantages that app ear in T able III. It ca n be s een that for L = 23 the average runtime is low for bo th a lgorithms. Th is s cenario represents a ca se that the value of | S | is very low compared to 2 N , and thus it is eas ier to find a partition s et D . Moreover , M-OMP become s faster as the solution is s parser . C. Ap plicability The p re v ious experiments de monstrated the app licability of SBR4 and SBR2 to s ignals that lie in M . W e now explore the case in which x ( t ) / ∈ M . In this expe riment we used the basic setup with L = 199 but the signals are con structed in a dif feren t way . Each one of the 100 0 signals is co nstructed by X ( f ) = α ( f ) ( S R ( f ) + j S I ( f )) , ∀ f ∈ F 0 . The function α ( f ) d epends on the algorithm and it makes sure that x ( t ) ∈ A K for the test case s of SBR4. Simil a rly , α ( f ) is used to form signals x ( t ) ∈ B K ∩ A 2 K for SBR2. The con struction of these signals de pends on L beca use of the de finitions of A K and B K . W e se lected K = 8 which results in a Landau rate of K /LT = 804 MHz in either con struction. In addition, we made sure that the s ignals do not li e in M . Fig. 6 s hows the empirical recovery rate of S BR4 and SBR2 in this scena rio. The value p = 4 N = 16 se rves a s a thresh old for sa tisfactory recovery , as the sampling rate for this value of p is p/LT = 1608 MHz, which is twice the La ndau rate. It can also be s een that SBR4 performs b etter than SBR2 as it does n ot in volve a s ub-optimal stag e of recovering the pa rtition set D . Both algorithms suf fer from the sub-optimality tec hniques for MMV systems. Note that the signals here a re synthesize d so that they lie in the relevant sets. Howe ver , for a gene ric signa l x ( t ) / ∈ M there is n o way to kno w in advance wh ether it li e s in one of these s ets. Moreover , there is no way to infer it from the samples , ( ¯ y )( f ) . In addition, even if SBR4 is used for this signal and it returns flag=1, there is no meaning for this indication s ince the uniqu eness of the solution is guaranteed on ly for x ( t ) ∈ A K which cannot be ensu red for a ge neric multi-band signa l. D. Ra ndom sampling pa tterns Theorem 3 requires a universal sampling pattern, which means finding a pattern res ulting in σ ( ( ¯ A )) = p . Howe ver , c omputing the v a lue of σ ( ( ¯ A )) requires a co mbinatorial p rocess for n on-prime L . The ”bunche d” pattern 26 5 10 15 20 25 30 0 0.2 0.4 0.6 0.8 1 p/N Empirical success rate SBR4 SBR2 Fig. 6. Performance for signals x ( t ) not in M . 5 10 15 20 25 30 0 0.2 0.4 0.6 0.8 1 p Empirical success rate SBR4 SBR2 Fig. 7. Performance of S BR algorithms wit h L = 200 . C = { 0 , 1 , ..., p − 1 } g i ven in [8] is proved to be univ e rsal but the matrix ( ¯ A ) is not well conditioned for this choice [7]. Alt e rnati vely , it follows from the work o f Cand ` es e t. al. [19] tha t rando m s ampling patterns a re mo st likely to produc e a high value for σ ( ( ¯ A )) if L, p are lar g e enough. Therefore, for practical us age the sa mpling pattern can be selected randomly even for non-prime L . Fig. 7 presen ts a n experiment with L = 200 . W e point out that the ran dom selection p rocess is carried out only o nce, and the same sampling patterns are us ed for all the tested signals. Comparing Figs. 4 and 7 it is seen that the resu lts are very similar a lthough the exa ct value of σ ( ( ¯ A )) is unknown. This experiment was also performed when for every N ≤ p ≤ 8 N the pa tterns are selected as C = { c k | c k = 2 k , 0 ≤ k ≤ p − 1 } , which is proved in [10] to rend er σ ( ( ¯ A )) = 1 . In this case both SBR 4 and SBR2 could n ot recover any of the 1 000 test ca ses. Th us, the uni versality of the pattern is crucial to the succe ss of our metho d. V I I I . C O N C L U S I O N S In this pap er we su ggested a me thod to reconstruct a multi-band signa l from its samples when the band locations a re unknown. Our development enab les a fully spec trum-blind s ystem where bo th the sampling a nd the reco nstruction stag es do not req uire this kn owl e dge. 27 Our main co ntrib ution is in proving that the reconstruction p roblem c an be formulated as a finite dimensional problem within the framework o f c ompressed s ensing. This resu lt is a ccomplished withou t a ny disc retization. Conditions for un iqueness of the solution and a lgorithms to find it were developed ba sed on known theoretical results and algo rithms from the CS literature. In add ition, we proved a lower bound on the sampling rate that improves o n the La ndau rate for the cas e of spectrum-blind reconstruction. One of the algorithms we propose d indeed approach es this minimal rate for a wide class of multi -ba nd signals ch aracterized by the n umber of bands and their widths . Numerical experiments d emonstrated the trade off b etween the av erag e sa mpling rate a nd the empirical succe ss rate of the re construction. A P P E N D I X A R E A L - V A L U E D S I G N A L S In order to treat real-v alue d signal the follo wing defin itions replace the ones given in the pa per . Th e class M is cha nged to contain all real-valued multi-band signals ba ndlimited to F = [ − 1 / 2 T , 1 / 2 T ] with no more than N bands on both sides of the spec trum, where each the band width is upper bounde d by B a s before. N ote tha t N is ev e n as the Fourier transform is conjugate symmetric for rea l-v alue d signa ls. The Nyquist rate remains 1 /T an d the Lan dau rate is N B . Repeating the c alculations of [8] that lead to (15) it can be seen that several mod ifications are required a s now explained. T o form ( ¯ x )( f ) , the interval F is still d i vide d into L equa l intervals. Howev e r , a slightly d if ferent treatment is given for odd a nd even values of L , becau se of the negative side of the spectrum. Define the set of L con secutive integeres K =           − L − 1 2 , · · · , L − 1 2  , odd L  − L 2 , · · · , L 2 − 1  , ev e n L . and redefin e the interv al F 0 F 0 =           − 1 2 LT , 1 2 LT  , odd L  0 , 1 LT  , ev e n L . The vector ( ¯ x )( f ) is no w defi ned a s ( ¯ x ) i ( f ) = X ( f + K i /LT ) , ∀ 0 ≤ i ≤ L − 1 , 28 The dimension s of ( ¯ A ) remain p × L with ik entry ( ¯ A ) ik = 1 LT exp  j 2 π L c i K k  , (52) 1 ≤ i ≤ p, 0 ≤ k ≤ L − 1 . The de finition of ( ¯ y )( f ) remains the s ame with res pect to F 0 defined here. The res ults of the p aper are thus extended to real-valued multi-band sign als sinc e (16) is no w valid with respe ct to these definitions of ( ¯ x )( f ) , ( ¯ A ) , and F 0 . Note tha t, we co uld have, c onceptua lly , cons tructed a c omplex-v a lued multi-band s ignal by taking only the positiv e frequencies of the real-valued signal. The Lan dau rate of this complex version is N B / 2 . Nevertheless, the information rate is the same as eac h samp le o f a complex-valued signal is rep resented by two real n umbers. A P P E N D I X B P R O O F O F L E M M A 1 Let r = rank( ( ¯ P )) . Reorde r the c olumns of ( ¯ P ) so that the first r c olumns are linearly ind epende nt. This operation does no t change the rank of ( ¯ P ) no r the rank of ( ¯ A ) ( ¯ P ) . De fine ( ¯ P ) = [ ( ¯ P ) (1) ( ¯ P ) (2) ] , (53) where ( ¯ P ) (1) contains the first r columns of ( ¯ P ) and the rest a re c ontained in ( ¯ P ) (2) . Therefore, r ≥ rank( ( ¯ A ) ( ¯ P )) = r ank( ( ¯ A )[ ( ¯ P ) (1) ( ¯ P ) (2) ]) ≥ rank( ( ¯ A ) ( ¯ P ) (1) ) . The ine qualities resu lt from the prop erties o f the ran k of conc atenation and of multiplication of matrices. So it is sufficient to p rove that ( ¯ A ) ( ¯ P ) (1) has full column rank. Let α be a vector of c oefficients so that ( ¯ A ) ( ¯ P ) (1) ( ¯ α ) = ( ¯ 0) . It rema ins to prove that this implies ( ¯ α ) = ( ¯ 0) . Denote k = | I ( ( ¯ P )) | . Since I ( ( ¯ P ) (1) ) ⊆ I ( ( ¯ P )) = k the vector ( ¯ P ) (1) ( ¯ α ) is k -s parse. Howev er , σ ( ( ¯ A )) ≥ k and its null spa ce ca nnot contain a k -sparse vec tor unles s it is the zero vector . S ince ( ¯ P ) (1) contains linearly inde penden t columns this implies ( ¯ α ) = ( ¯ 0) . A P P E N D I X C C O M P U T A T I O N O F T H E M AT R I X ( ¯ Q ) The SBR4 algorithm compu tes the matrix ( ¯ Q ) in the frequency d omain. A me thod to co mpute this matrix directly from the samples in the time domain is no w p resented. Consider the ik th e lement of ( ¯ Q ) from (33): ( ¯ Q ) ik = Z 1 LT 0 ( ¯ y ) i ( f ) ( ¯ y ) ∗ k ( f ) d f . (54) 29 Since ( ¯ y ) i ( f ) is the DTFT of x c i [ n ] we can write ( ¯ Q ) ik as, ( ¯ Q ) ik = Z 1 LT 0 X n i ∈ Z x c i [ n i ] exp ( − j 2 π f n i T ) ! · (55) X n k ∈ Z x c k [ n k ] exp ( − j 2 π f n k T ) ! ∗ d f = X n i ∈ Z X n k ∈ Z x c i [ n i ] x ∗ c k [ n k ] Z 1 LT 0 exp ( j 2 π f ( n k − n i ) T ) d f . Note that from (14) the seque nce x c i [ n i ] is padded by L − 1 z eros be tween the non-zero samples . D efine the sequen ce without thes e ze ros as ˆ x c i [ m ] = x ( mLT + c i T ) , m ∈ Z , 1 ≤ i ≤ p. (56) Then, (55) can be written as ( ¯ Q ) ik = X m i ∈ Z X m k ∈ Z ˆ x c i [ m i ] ˆ x ∗ c k [ m k ] g ik [ m i − m k ] (57) = X m i ∈ Z ˆ x c i [ m i ]( ˆ x c k ∗ g ik )[ m i ] , where g ik [ m ] = Z 1 LT 0 exp ( j 2 π f ( mL + ( c k − c i )) T ) d f , (58) and ( ˆ x c k ∗ g ik )[ m ] = X n ∈ Z ˆ x ∗ c k [ n ] g ik [ m − n ] . (59) If i = k then c i = c k and g ii [ m ] = g [ m ] = 1 LT exp( j π m ) sinc( m ) , (60) with sinc( x ) = sin ( π x ) / ( π x ) . If i 6 = k , g ik [ m ] = exp  j 2 π L ( c k − c i )  − 1 j 2 π ( mL + ( c k − c i )) T . (61) The set of digital filter g ik can be designed immediately after setting the para meter L, p, C as the se filters do not dep end on the signa l. R E F E R E N C E S [1] H. Landau, “Necessary density conditions for sampling and interpolation of certai n entire functions, ” Acta Math., 1967. [2] R. G. V aughan, N. L . S cott, and D. R. White, “The t heory of bandpass samp li ng, ” IEEE T rans. Signal Pr ocessing , vo l. 39, pp. 1973-198 4, S ept. 1991. 30 [3] A. K ohlenberg, “Exact i nterpolation of band-limited functions, ” J . Appl. Phys., vol. 24, pp. 1432-1435, Dec 1953. [4] Y . -P . Lin and P . P . V aidyan athan, “Periodically nonu niform sampling of bandpass signals, ” IEEE T rans. Cir cuits Syst. II, vol. 45, pp. 340-351, Mar . 1998. [5] C. Herley and P . W . W ong , “Minimum rate sampling and reconstructtion of si gnals w ith arbitrary frequency support” IE EE Tr ans. Inform. T heory , vol. 45, pp. 1555-1564, July 1999. [6] P . Feng and Y . Bresler , “Spectrum-blind minimumrate sampling and reconstruction of multi-band signals, ” in Pr oc. IEEE Int. Conf. ASSP , Atlanta, GA, IEEE, May 1996. [7] Y . Bresler and P . Feng, “S pectrum-blind minimumrate sampling and reconstruction of 2-D multi-band signals, ” in Pr oc. IEEE Int. Conf. Image Pr oc. , Lausanne, Switzerland, IEEE , Sept. 1996. [8] R. V enkataramani and Y . Bresler, “Perfect reconstruction formulas and bounds on aliasing error in sub-Nyquist nonuniform sampling of multiband signals, ” IEEE T rans. Inform. Theory , vol. 46, pp. 2173-2183 , Sept. 2000. [9] E. J. Cand ` es, J. Romberg and T . T ao. “Robust uncertainty principles: ex act signal reconstruction from highly incomplete frequency information, ” IE EE Tr ans. Inform. Theory , vol. 52, pp. 489-50 9. [10] M. Mishali, Y . C. El dar and M. Horowitz “Properties of univ ersal sampling patterns for multi-band signals, ” IEEE Lett ers. on Signal Pr ocess., submitted for publication, Mar . 2007. [11] D. L. Donoho and M. Elad, “Ma ximal sparsity representation via ℓ 1 minimization, ” Pr oc. Natl . Acad. Sci., vol. 100, pp. 2197-2202, Mar . 2003. [12] G. Dav i s, S . Mallat, and M. A vellaned a, “ Adaptive greedy approximations, ” J. Constructive Appr ox., vol. 13, no. 1, pp. 5798, 1997. [13] D. L. Donoho, “Compressed Sensing, ” IEEE T rans. Inform. Theory , vol. 52, Iss.4, pp. 1289–1306, April 2006. [14] J. B . K ruskal, “T hree-way arrays: rank and uniqueness of trili near decompositions, with application to arithmetic complexity and statistics, ” Linear Alg. Its Applic., v ol. 18, no.2, pp. 95-138, 1977. [15] J. Chen, and X. Huo, “Theoretical results on sparse representations of multi ple measurement vectors, ” IEEE T rans. Signal Pr ocess, vol. 54, pp. 46344643, Dec. 2006. [16] S . F . Cotter , B . D . Rao, K. Engan, and K. K reutz-Delgado, “Sparse solutions to linear in verse problems with multiple measurement vectors, ” IEEE T rans. Signal Pr ocess, vol. 53, pp. 24772488, Jul. 2005. [17] S . Chen and D. Donoho, “Basis pursuit, ” in Proc. T wenty-Eighth A silomar Conf. Signals, Syst., Comput., vol. I, Monterey , CA, Nov . 1994, pp. 4144. [18] S . G. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries, ” IEEE Tr ans. Signal Pr ocess., vol. 41, no. 12, pp. 339734 15, Dec. 1993. [19] E . J. Cand ` es, J. Romberg and T . T ao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequenc y information, ” IE EE Tr ans. Inform. Theory , vol. 52, pp. 489-50 9. [20] Y . M. Lu and M. N. Do, “ A theory for sampling signals from a union of subspaces, ” IEEE T rans. Signal Process, submitted for publication, 2007. [21] J. Trop p, M. W akin, M. Duarte, D. Baron and R. Baraniuk, “Random filters for compressi ve sampling and reconstruction, ” Pr oc. IEEE Int. Conf. on Acoustics, Speech, and Signal Proc essing (ICASSP) , T oulouse, Fr ance, May 2006. [22] J. Laska, S .Kirolos, M. Duarte, T . Ragheb, R . Baraniuk and Y . Massoud , “Theory and implementation of an analog-to-information con verter using random demodulation, ” Pr oc. IEEE Int. Symp. on Ci r cuits and Systems (ISCAS) , New Orleans, Louisiana, 2007.