Deformed Statistics Free Energy Model for Source Separation using Unsupervised Learning

Deformed Statistic s Free Ener gy Model f or Source Separation using Unsupervised Lear ning R. C. V enkates an Systems Research Corp oration Aundh, Pune 4 1100 7, India Email: ravi@systemsresearchcor p.com A. Plastino IFLP , Nation al University La Plata & National Research Council (CONICET) C. C., 727 19 00, La Plata,Argen tina Email: plastino @venus.fisica.unlp.e du.ar Abstract —A generalized-statistics variational principle f or source separa tion is formulated by r ecourse to Tsa llis’ entr opy subjected to the additiv e duality and employing constraints described by normal a vera ges. Th e variational principle is amalgamate d with Hopfield-like learning rules resulting in an unsupervised learning model. The up date rules are fo rmulated with the aid of q -deform ed calculus. Numerical examples exem- plify the efficacy of t his model. I . I N T R O D U C T I O N Recent studies hav e suggested th at minimization of th e Helmholtz free energy in statistical ph ysics [1] plays a ce ntral role in un derstandin g action, per ception, and learnin g (see [2] and the referenc es therein). In fact, it has be en sugg ested that the principle of f ree energy m inimization is e ven m ore fundam ental than the redund ancy reduction prin ciple (also known as the principle of ef ficient cod ing) articu lated by Barlow [3] and later for malized by Linsker as the In fomax principle [4 ]. Specifically , the principle of efficient coding states that the brain should op timize the mu tual info rmation between its sensory sign als and some parsimo nious neuronal representatio ns. This is identical to optimizing the param eters of a genera tiv e model to maximize the accu racy of predictions, under complexity constrain ts. Both are mandated by the f ree- energy principle, which can be regarded as a proba bilistic generalizatio n of the In fomax principle. The Infomax p rinciple has bee n centr al to the develop- ment of in depend ent compo nent analy sis (ICA) and th e allied problem of b lind source separation (BSS) [ 5]. W ithin the ICA/BSS co ntext, very f ew models based on minim ization of the free energy exist, the most p rominen t of them originated by Szu and co-workers (eg. see Refs. [ 6,7]) to achieve source separation in remote sensing (i.e. hypersp ectral imaging (HSI)) using the ma ximum entr opy pr inciple. The ICA/BSS p roblem may be summ arized in term s of the relatio n As = x , (1) where s is the unkn own sou rce vector to be extracted , A is the unkno wn mixing matr ix (also known as reflectance matrix or material abundance matrix in HSI) , a nd x is the known vector of observed data. The Helmholtz free energy is described within the framework of Boltzmann-Gib bs-Shanno n (B-G-S) statistics as F ( T ) = U − k B T S, (2) where T is the ther modyn amic tem perature (or haemo static temperatur e in the parla nce of cyber netics), k B the Boltzmann constant, U the intern al en ergy , and S Shanno n’ s en tropy . A more prin cipled and systematic manner in which to study free energy minimization within the context of the ma ximum entropy principle (MaxE nt) is by substitutin g the minimizatio n of the Helmholtz free e nergy princ iple with the max imizing of the M assieu potential [8] Φ( β ) = S − β U, (3) where β = 1 k B T is the inverse thermo dynam ic temp erature. The Massieu potential is the Legen dre transform of the Helmholtz free energy , i.e.: Φ ( β ) = − F ( T ) T . The g eneralized (also, interchangeab ly , nonadditive, de- formed , or no nextensiv e) statistics of Tsallis ha s r ecently been the f ocus of much attentio n in statistical p hysics, complex sys- tems, and allied disciplines [ 9]. No nadditive statistics su itably generalizes the extensi ve, orthodox B-G-S one. The scope of Tsallis statistics has lately been extended to stud ies in lossy data compr ession in communication theo ry [10] and mach ine learning [11, 12]. It is important to note that po wer law distrib utions lik e the q- Gaussian distribution can not be accurately mod eled within th e B-G-S framework [9]. One of the most commonly encountered source of q-Gaussian distributions o ccurs in the proce ss of normalizatio n of m easurement da ta using Stud entization tech- niques [13]. q-Gaussian beh avior is also exhibited by ellip- tically inv ar iant d ata, wh ich genera lize spherically symmetric distributions. q-Gaussian’ s are also an excellent appro ximation to corr elated Gau ssian d ata, and o ther impor tant and f unda- mental ph ysical and biological pro cesses (for example, see [14] and the references therein). This paper inten ds to accomp lish the following objectiv es: • ( i ) to formulate and solve a variational principle for source separation using the maximu m d ual Tsallis en- tropy with co nstraints de fined by norma l a verages expec- tations, • ( ii ) to amalgamate the variational principle with Hopfield-like lea rning rules [15] to acq uire inform ation regarding unknown p arameters via an unsup ervised learn- ing parad igm, • ( iii ) to formulate a nume rical fra mew ork for the gener- alized statistics unsu pervised learning model and demo n- strate, with the aid of n umerical examples for separation of indep endent sources ( en dmembers ), th e supe riority of the gen eralized statistics sourc e separation mo del vis- ´ a- vis an e quiv ale nt B-G-S m odel for a single p ixel. It is important to note that by amalgamating the inform ation- theoretic model with the Ho pfield model, [ A ] acqu ires th e role of the Associati ve Me mory (AM ) matrix . Fu rther , employ ing a Hop field-like learning rule r end ers the model presented in this paper r ead ily amenable to har dwa r e implementa tion using F ield P r ogrammable Gate Arrays (FPGA’ s) . The additive dua lity is a fund amental pr operty in gen er- alized statistics [9]. On e imp lication of the additive duality is that it p ermits a deform ed logar ithm defined by a giv en nonad ditivity p arameter (say , q ) to be inferre d from its d ual deformed log arithm p arameterize d by : q ∗ = 2 − q . This p aper derives a variational princip le for source sep aration using the dual Tsallis entropy using nor mal averages constrain ts. T his approa ch has been previously utilized (fo r eg. Ref. [16]), and possess the prop erty o f seamlessly yielding a q ∗ -defor med exponential form on variational extremization. An impo rtant issue to address con cerns the manner in which expectation values are co mputed . Of the various fo rms in which expectations may be defined in n onextensive statis- tics has, only the line ar constraints originally e mployed b y Tsallis [9] (also known as normal ave rag es ) of the form: h A i = P i p i A i , has b een found to be physically satisfactory and consistent with b oth th e gener alized H-theo rem an d the generalized Stosszahlansatz (mo lecular chao s hyp othesis) [17, 18]. A re-formulation of the variational pe rturbation appro xi- mations in non extensiv e statistical physics follo we d [1 8], v ia an application of q -defo rmed calcu lus [ 19]. Results from the study in Ref. [19] hav e been successfully utilized in Section IV of th is paper . This intr oducto ry Section is co ncluded b y b riefly describ ing the suitability of employing a generalized statistics model to study the so urce separation problem. First, in the case of remote sensing applications, and even mo re so in the case o f HSI, th e observed d ata are highly cor related, even in the case of a single p ixel. Next, the observed data are requir ed to be normalized (scaled ). The Studentization pr ocess is one of the most prom inent m ethods utilized to nor malize the observed data [20,21 ]. Both these features lead to an excursion f rom the Ga ussian fram ew o rk (B-G-S statistics) and r esult in q- Gaussian pdf ’ s characterized by the q -defo rmed e x ponen tial: exp q ( − x ) = [1 − (1 − q ) x ] 1 1 − q , wh ich maximizes the Tsallis entropy . I I . T H E O R E T I C A L P R E L I M I N A R I E S The Sectio n intro duces the essential co ncepts arou nd which this commun ication rev o lves. The Tsallis entro py is defined as [9] S q ( X ) = − P x p ( x ) q ln q p ( x ) . (4) The q -deformed lo garithm and th e q -deformed expo nential are defined as [9, 1 9] ln q ( x ) = x 1 − q − 1 1 − q , and, exp q ( x ) =  [1 + (1 − q ) x ] 1 1 − q ; 1 + (1 − q ) x ≥ 0 , 0; otherw ise (5) Note that as q → 1 , (4 ) acqu ires the fo rm of the equiva- lent B-G-S entro pies. Like wise in (5), ln q ( x ) → ln( x ) a nd exp q ( x ) → exp( x ) . The o peration s o f q -deformed relations are governed by q-d eformed algeb ra and q-d eformed calculu s [19]. Ap art from providing an an alogy to equiv alent expres- sions d erived from B-G-S statistics, q- deformed algeb ra and q-deformed calculus endow generalized statistics with a unique informa tion geometr ic structure . T he q-deformed ad dition ⊕ q and the q- deformed s ubtraction ⊖ q are defined as [19] x ⊕ q y = x + y + (1 − q ) xy, ⊖ q y = − y 1+(1 − q ) y ; 1 + (1 − q ) y > 0 ⇒ x ⊖ q y = x − y 1+(1 − q ) y (6) The q-de formed d eriv ative, is defined as [19] D x q F ( x ) = lim y → x F ( x ) − F ( y ) x ⊖ q y = [1 + (1 − q ) x ] dF ( x ) dx (7) As q → 1 , D x q F ( x ) → dF ( x ) /dx , the Newt onian der iv ative. The Leibnitz r ule for defo rmed deri vativ es [19] is D x q [ A ( x ) B ( x )] = B ( x ) D x q A ( x ) + A ( x ) D x q B ( x ) . (8) Re-parameter izing ( 5) via the a dditive d uality [10] : q ∗ = 2 − q , y ields the d ual deformed logarithm and exponen tial ln q ∗ ( x ) = − ln q  1 x  , and, exp q ∗ ( x ) = 1 exp q ( − x ) . (9) The dual Tsallis en tropy is defin ed by [10, 1 6] S q ∗ ( X ) = − X x p ( x ) ln q ∗ p ( x ) . (10) Here, ln q ∗ ( x ) = x 1 − q ∗ − 1 1 − q ∗ . The dua l Tsallis entr opies ac- quir e a form similar to th e B-G-S entr opies, with ln q ∗ ( • ) r epla cing ln( • ) . I I I . V A R I AT I O N A L P R I N C I P L E Consider the Lag rangian Φ q ∗ [ s j ] = − P j s j ln q ∗ s j − N P i =1 N P j =1 λ i ( A ij s j − x i ) + λ 0 N P j =1 s j − 1 ! , (11) subject to th e compon ent-wise co nstraints N X j =1 s j = 1 , and N X j =1 A ij s j = x i . (12) Clearly , the RHS of th e Lagr angian (11) is the q ∗ -defor med Massieu poten tial: Φ q ∗ [ λ ] , subject to the no rmalization con- straint on s j . T he variational extremization o f (1 1), perfor med using the Ferri- Martinez-Plastino methodo logy [22], leads to ⇒ − (2 − q ∗ ) (1 − q ∗ ) s 1 − q ∗ j − N P i =1 λ i A ij + λ 0 = 0 ⇒ s j =  (1 − q ∗ ) ( q ∗ − 2)  − λ 0 + N P i =1 λ i A ij  1 1 − q ∗ (13) Multiplying the seco nd relation in (13) by s j and summing over all j , yield s after ap plication of the nor malization condi- tion in ( 12) − (2 − q ∗ ) (1 − q ∗ ) ℵ q ∗ − N X j =1 N X i =1 λ i A ij s j = − λ 0 , (14) where: ℵ q ∗ = N P j =1 s 2 − q ∗ j , and substituting (14) in to the third relation in (1 3) yields s j = " ℵ q ∗ + (1 − q ∗ ) N P j =1 N P i =1 ˜ λ i A ij s j − (1 − q ∗ ) N P i =1 ˜ λ i A ij # 1 1 − q ∗ ; ˜ λ i = λ i (2 − q ∗ ) . (15) Eq. (15 ) y ields after so me algebra s j = exp q ∗  − N P i =1 ˜ λ ∗ i A ij  ℵ q ∗ +(1 − q ∗ ) N P j =1 N P i =1 ˜ λ i A ij s j ! 1 q ∗ − 1 , (16) where ˜ λ ∗ i = ˜ λ i ℵ q ∗ +(1 − q ∗ ) N P j =1 N P i =1 ˜ λ i A ij s j , and , ℵ q ∗ + (1 − q ∗ ) N P j =1 N P i =1 ˜ λ i A ij s j ! 1 q ∗ − 1 = ˜ Z q ∗ . (17) Here ˜ Z q ∗ is the can onical p artition f unction , w here: ˜ Z q ∗ = N P j =1 exp q ∗  − N P i =1 ˜ λ ∗ i A ij  . The dual Tsallis en tropy takes the form S q ∗ [ s ] = ℵ q ∗ − 1 ( q ∗ − 1) ; N P j =1 s j = 1 ⇒ ℵ q ∗ = 1 + ( q ∗ − 1) S q ∗ [ s ] (18) Substituting n ow (18) into th e expr ession for : ˜ Z q ∗ in (17) results in − ln q ∗  1 ˜ Z q ∗  = S q ∗ [ s ] − N P j =1 N P i =1 ˜ λ i A ij s j = Φ q ∗ h ˜ λ i . (19) Clearly , Φ q ∗ h ˜ λ i in (19) is a q ∗ -defor med Massieu poten tial. By substituting (18 ) into (14 ) we arrive at S q ∗ [ s ] − N P j =1 N P i =1 ˜ λ i A ij s j = − ˜ λ 0 + 1 (1 − q ∗ ) = ˆ λ 0 ; ˜ λ 0 = λ 0 (2 − q ∗ ) . (20) Again, ˆ λ 0 in (20) is a q ∗ -defor med Massieu p otential: Φ q ∗ [ ˜ λ ] . W e wish to relate ˆ λ 0 and ˜ Z q ∗ . T o this en d, comparison of (19) and (20 ) y ields ˆ λ 0 = − ˜ λ 0 + 1 (1 − q ∗ ) = − ˜ Z q ∗ − 1 q ∗ (1 − q ∗ ) + 1 (1 − q ∗ ) ⇒ ˜ Z q ∗ = h (1 − q ∗ ) ˜ λ 0 i 1 q ∗ − 1 ; ˜ λ 0 = λ 0 (2 − q ∗ ) , (21) so th at, by substituting (18) into (15) an d the n in voking (20) we get s j =  1 − (1 − q ∗ )  N P i =1 ˜ λ i A ij + ˆ λ 0  1 1 − q ∗ ; ˆ λ 0 = − ˜ λ 0 + 1 (1 − q ∗ ) . (22) Here, (22) is re-defined with the aid of (20 ) as s j =  1 − (1 − q ∗ ) N P i =1 ˜ λ ∗ i A ij  1 1 − q ∗ [ 1 − (1 − q ∗ ) ˆ λ 0 ] 1 q ∗ − 1 =  1 − (1 − q ∗ ) N P i =1 ˜ λ ∗ i A ij  1 1 − q ∗ ˜ Z q ∗ ; where ˜ λ ∗ i = ˜ λ i 1 − (1 − q ∗ ) ˆ λ 0 , ˜ Z q ∗ = N P j =1  1 − (1 − q ∗ ) N P i =1 ˜ λ ∗ i A ij  1 1 − q ∗ . (23) W ith the aid of ( 21), (22) is re-cast in the f orm s j = exp q ∗  − N P i =1 ˜ λ ∗ i A ij  [ (1 − q ∗ ) ˜ λ 0 ] 1 q ∗ − 1 ; where , ˜ λ i = λ i (2 − q ∗ ) , ˜ λ 0 = λ 0 (2 − q ∗ ) , ˜ λ ∗ i = ˜ λ i [ (1 − q ∗ ) ˜ λ 0 ] . (24) Finally , inv oking the nor malization of s j , (24) yield s h (1 − q ∗ ) ˜ λ 0 i 1 q ∗ − 1 = N X j =1 " 1 − (1 − q ∗ ) N X i =1 ˜ λ ∗ i A ij # 1 1 − q ∗ . (25) Note the self-r eferential natur e of (2 3) in the sense that: ˜ λ ∗ i (defined in (20) a nd ( 23) is a f unction of ˜ λ 0 . T he Lagran ge multiplier ˜ λ ∗ i is hen ceforth define d in this paper as the d ual normalized Lagrange force mu ltiplier . I V . U N S U P E RV I S E D L E A R N I N G R U L E S The pr ocess o f unsup ervised learn ing is amalgamated to the above information theoretic structure via a Ho pfield-like learning ru le to update the AM m atrix [ A ] in the case of a perturb ation ∆ x j of the obser ved d ata dx j dt = ∂ ˜ Φ ∗ q ∗ [ s j ] ∂ s j = − 1 − (1 − q ∗ ) ˜ λ (1 − q ∗ ) ˜ λ 0 − ln q ∗ s j ˜ λ 0 − (1 − q ∗ ) N P i =1 ˜ λ ∗ i A ij ⇒ ∆ x j = −  1 − (1 − q ∗ ) ˜ λ 0 (1 − q ∗ ) ˜ λ 0 + ln q ∗ s j ˜ λ 0 + (1 − q ∗ ) N P i =1 ˜ λ ∗ i A ij  ∆ t ; whe re, ˜ Φ ∗ q ∗ [ s j ] = Φ q ∗ [ s j ] (2 − q ∗ ) ˜ λ 0 , (26) which is o btained from the first relation in (13 ) and (24) . Gradient ascent alo ng with (24 ) originates the seco nd learning rule dx j dt = ∂ Φ ∗ q ∗ [ s j ] ∂ A ij = − ˜ λ ∗ i s j ⇒ ∆ x j = −  ˜ λ ∗ i s j  ∆ t ; whe re, Φ ∗ q ∗ [ s j ] = Φ q ∗ [ s j ] (1 − q ∗ ) ˜ λ 0 . (27) In (26) and ( 27), Φ q ∗ [ s j ] is the LHS of the La grangia n (11 ). Now , a critical upda te rule is that fo r the chang e in the dual normalized Lagrange for ce multipliers ˜ λ ∗ i resulting from a pertur bation ∆ x j in the observed data. Usua lly (as stated within the con text of the B-G-S fra mew ork), such an update would entail a T aylor-expansion yielding u p to the first order : ∆ x j = N P k =1 ∂ x j ∂ ˜ λ ∗ k ∆ ˜ λ ∗ k . Such an analysis is valid only for distributions cha racterized by th e regular expon ential exp ( − x ) . For probability distributions character ized by q - deform ed expo nentials, i.e., the on es we face her e, such a perturb ation treatment would lead to u n-phy sical results [18] . Thus, following the prescr iption given in Ref. [18], for a function : F ( τ ) = P n F ( τ n ) the chain rule yields: dF ( τ ) d ˜ λ ∗ k = dF ( τ ) dτ dτ d ˜ λ ∗ k . Thus, replacin g the Newtonian deriva- ti ve: dF ( τ ) dτ by the q ∗ -deformed o ne defined b y (7) ( see Ref. [19] ): D τ q ∗ F ( τ ) = [1 + (1 − q ∗ ) τ ] dF ( τ ) dτ and defining : D τ q ∗ F ( τ ) dτ d ˜ λ ∗ k = δ q ∗ ,τ F ( τ ) as well, facilitates the desired transform ation: dF ( τ ) d ˜ λ ∗ k → δ q ∗ ,τ F ( τ ) . Conseq uently , the update rule for ˜ λ ∗ k is r e-form ulated via q -deform ed calcu lus in the fashion ∆ x j = N X k =1 " D τ q ∗ N X i =1 A j i s i # ∆ ˜ λ ∗ k = N X k =1 " N X i =1 D τ q ∗ A j i s i # ∆ ˜ λ ∗ k . (28) Additionally , setting: − A ik ˜ λ ∗ k = τ in (23) leads to s j = [1 + (1 − q ∗ ) τ ] 1 1 − q ∗ ˜ Z q ∗ . (29) Employing at this stage th e Leibn itz rule for q ∗ -defor med deriv atives (and rep lacing q by q ∗ in (8)) , th e term within square par enthesis RHS in (28) yields N P i =1 D τ q ∗ A j i s i = N P i =1 n A ji ˜ Z q ∗ D τ q ∗ [1 + (1 − q ∗ ) τ ] 1 1 − q ∗ + A j i [1 + (1 − q ∗ ) τ ] 1 1 − q ∗ D τ q ∗  1 ˜ Z q ∗ o , (30) a relation that, after expa nsion tur ns into N P i =1 D τ q ∗ A j i s i = N P i =1 n A ji ˜ Z q ∗ [1 + (1 − q ∗ ) τ ] ∂ τ ∂ ˜ λ ∗ k ∂ ∂ τ [1 + (1 − q ∗ ) τ ] 1 1 − q ∗ + A j i [1 + (1 − q ∗ ) τ ] 1 1 − q ∗ D τ q ∗  1 ˜ Z q ∗ o = N P i =1 n − A ji ˜ Z q ∗ [1 + (1 − q ∗ ) τ ] 1 1 − q ∗ A ik − A j i [1 + (1 − q ∗ ) τ ] 1 1 − q ∗ [1 + (1 − q ∗ ) τ ] ∂ τ ∂ ˜ λ ∗ k ˜ Z − 2 q ∗ ∂ ˜ Z q ∗ ∂ τ o = − N P i =1 A j i s i A ik + N P i =1 A j i [1+(1 − q ∗ ) τ ] 1 1 − q ∗ ˜ Z q ∗ N P k =1 A ik ˜ Z q ∗ [1 + (1 − q ∗ ) τ ] 1 1 − q ∗ = − N P i =1 A j i s i A ik + x j x k . (31) Finally , the update r ule for ˜ λ ∗ k with respect to ∆ x j adopts th e appearan ce ∆ x j = N X k =1 x j x k − N X i =1 A j i s i A ik ! ∆ ˜ λ ∗ k . (32) V . N U M E R I C A L C O M P U TA T I O N S The proced ure for our doub le recu rsion p roblem is su mma- rized in the pseudo-code below Algorithm 1 Gen eralized Statistics Sou rce Separation Mod el (1 . ) Input : ( i ) . Observed d ata: x , ( ii ) . Trial values of dual norm alized Lag range force multipliers: ˜ λ ∗ , ( iii ) . Dual nonad ditiv e parame ter: q ∗ . (2 . ) Initializatio n : Obtain A (0) ij from: A (0) ij = x i σ q ∗ ( x j ) + 5 0 % r andom noise to bre ak any rank-1 singu larity . The q ∗ -defor med sigmoid logistic functio n is: σ q ∗ ( x j ) = 1 1+exp q ∗ ( − x i ) . (3 . ) First Recursion ( i ) Compu te: ˜ Z (0) q ∗ from (2 3), ( ii ) Comp ute: ˆ λ (0) from (21) , ( iii ) Compu te: s (0) j , ˜ λ (0) i , and ˜ λ (0) 0 from (23) /(24), ( iv ) Compute: x (0) i from ( 5), thus: ∆ x (0) j = x K now n j − x (0) j , ( v ) Co mpute ∆ ˜ λ ∗ (0) k by inv erting (3 2), ( v i ) Comp ute next estimate: ˜ λ ∗ (1) k = ˜ λ ∗ (0) k + ∆ ˜ λ ∗ (0) k . (4 . ) Second Recursion ( v ii ) Compute imp roved estimate of : A (1) ij from (26) by setting ∆ t = 1 an d solving : ∆ x j = −  1 − (1 − q ∗ ) ˜ λ (0) 0 (1 − q ∗ ) ˜ λ (0) 0 + ln q ∗ s (0) j ˜ λ (0) 0 + (1 − q ∗ ) N P i =1 ˜ λ ∗ (1) i A (1) ij  . (5 . ) Go to (3 . ) Follo wing the p rocedu re outlined in the above pseudo- code, values of ˜ λ ∗ = [0 . 6228 , 0 . 6337 , 0 . 4577 , 0 . 1095 , 0 . 7252 , 0 . 0 1752 , 0 . 4 128] and x = [0 . 5382 , 0 . 1023 , 0 . 6404 , 0 . 4358 , 0 . 0278 , 0 . 2425 , 0 . 3299] are provided. The se v alu es are the same as those in Ref. [ 7] and co nstitute experimentally obtained Lan dsat data for a single pixel. The d ifference between the generalize d statistics model presented in this paper and the B-G-S mo del of [6,7] lies in the fact that the form er has initial inp uts o f ˜ λ ∗ i ’ s, whereas the latter merely has initial inputs of λ ’ s (a far simpler case). The self-rer entiality in (23 ) mandates use of ˜ λ ∗ i ’ s as th e p rimary op erational Lagrange mu ltiplier . No te that the correlation coefficient of x K now n is unity , a sign ature of high ly correlated data. A value of q ∗ = 0 . 75 is cho sen. Figure 1 and Figu re 2 depict, vs. the n umber of iterations, the sour ce separ ation for th e g eneralized statistics model and for th e B-G-S mode l, respectively . V alues of x are denoted b y ”o”’ s. It is rea dily apprec iated that the gener alized statistics exhibits a more p ronou nced source separatio n than the B-G- S model. Owing to the highly correlated nature o f th e observed data, such results are to be expected . V I . S U M M A RY A N D D I S C U S S I O N A generalize d statistics model for source separation that em- ploys an unsupervised learning paradigm has been presented in this co mmun ication. This model is sho wn to exhibit superior separation perfor mance as comp ared to an equivalent model derived within the B-G-S fra mew ork. Our encou raging results should inspir e futu re work stud ies on the implicatio ns of first- order and second- order phase transitions of the Massieu pote n- tial. One would wish for a self-consistent scheme e nabling one to obtain self-consistent values of Lag range multipliers based on the principle of p hase transitions an d symmetry brea king. 0 10 20 30 40 50 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Generalized Statistics Model − Endmember Percentage Number of Iterations Fig. 1. Source separat ion for genera lized statistics model A C K N O W L E D G M E N T RCV grate fully acknowledges supp ort from RAND-MSR contract CSM-DI & S- QIT - 10115 5-03 -2009 . R E F E R E N C E S [1] R. Kubo, Stati stical Mechanic s ,2nd ed., Spri nger, Berlin, 2001. [2] K. Friston, ”The free-ene rgy pr incipl e: a unified brain theory?”, Nat. Rev . Neur osc., , 388 , 12, 2337, 2009. [3] H. Barlow , ”Possible principles underlying the transformations of sensory messages”. In Rosenblit h, W .. Sensory Communication , 217, Cambridge MA: MIT Press, 1961. 0 10 20 30 40 50 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Boltzmann−Gibbs−Shannon Model − Endmember Percentage Number of Iterations Fig. 2. Source separat ion for Boltzmann-Gibb s-Shannon m odel [4] R. Linsker , ”Perceptu al neu ral orga nisatio n: some approache s based on netw ork models and information theory”, A nnu. Rev . Neur osci. , 13 , 257, 1990. [5] A. J. Bell, T .J. Sejno wski, ”An information-ma ximizat ion approach to blind separation and blind decon voluti on”, Neural Comput. , 7 , 1129, 1995. [6] A. J. Bell, T .J. Sejno wski, ”An information-ma ximizat ion approach to blind separation and blind decon voluti on”, Neural Comput. , 7 , 1129, 1995. [7] H. H. Szu, C. Hsu, ”Landsat spectral Unm ixing a‘ la superresol ution of blind mat rix inv ersion by constraint MaxEnt neural nets”, Pr oc. SPIE 3078 , 147, 1997. [8] H. B. Callen, Thermod ynamics and an Intr oduction to T hermostati stics , 2nd ed., W iley , New Y ork, 1985. [9] C. T sallis, Intr oduction to Nonext ensive Statistical Mechani cs: Appr oach- ing a Complex W orld , Springer , Berlin, 2009. [10] R. C. V enkatesan and A. Plastino, ”Genera lize d statistic s framew ork for rate distortio n theory”, Physica A, , 388 , 12, 2337, 2009. [11] R. C. V enkatesan and A. Plastino, ”Deformed statistic s formulation of the informati on bottlenec k method”, Proc eedings of the IE EE Int. Symp. on Information Theory , 1323, IEE E Press, 2009;R. C. V enkatesan and A . Plastino, ”General ized statistic s frame work for the information bottlenec k method”, in preparation, 2011. [12] A. F . T . Martins, N. A. Smith, E. P . Xing, P . M. Q. Aguiar , and M. A. T . Figueire do , ”Nonext ensi ve informat ion theoreti c kernels on m easures”, JMLR, , 10 , 935, 2009. [13] C. V ignat and A . Plastino , ”Why is the detection of q-Gaussian behav ior such a common occurrence?” , Physica A , 388 , 601, 2009. [14] R. Hanel, S. Thurner , and C. Tsallis, ”Limit distrib utions of scale- in vari ant probabil istic m odels of corre lated random va riable s with the q-Gaussian as an explicit example” , Eur . Phys. J. B , 72 , 2, 263, 2009. [15] J. J. Hopfield, ”Neural network and physical systems with emergent colle cti ve computational abili ties”, Proc. Natl. Acad. Sci. 79 , 2554, 1982. [16] T . W ada and A. M. Scarfone , ”A Non self-referent ial expression of Tsallis’ probability distrib ution functio n”, Eur . Phys. J. B , 47 , 557, 2005. [17] S. Abe, ”Generalize d molecular chaos hypothesi s and the H theorem: Problem of constrai nts and a mendment of nonext ensi ve statistical me- chanic s”, Phys. Rev . E . [18] R. C. V enkatesan and A. Plast ino, ”General ized statistics v ariational perturba tion approximatio n using q-deforme d calcu lus”, Physica A, , 398 , 6, 1159, 2010. [19] E . Borges, ”A possible deformed algebra and calculus inspired in none xtensi ve thermostati stics”, Physica A 340 95, 2004. [20] R. A. Schowen gerdt, Remote Sensing: Models and Methods for Imag e Pr ocessing , 3rd ed., Academic Press, 2006. [21] P . K. V arshne y and M. K. Arora (Eds.), Advanc ed Image Pr ocessing T echnique s for Remotely Sensed Hyperspect ral D ata , Springer-V erlag, Berlin, 2004. [22] G. L . Ferri, S. Martinez and A. Plastino, ”Equi v alence of the four versi ons of T sallis sta tistic s”, J . Stat . Mec h.: Theory an d Experimen t 2005(04) P04009, 2005.