Privacy-Aware Guessing Efficiency

Pri v ac y-A wa re Guessing Ef ficienc y Shahab Asoodeh , Mario Diaz, Fady Alajaji, and T amás Lind er Abstract —W e in vestigate the pr oblem of guessing a discrete random variable Y under a priva cy constraint dictated by another correlated d iscrete random variable X , where both guessing efficiency and priv acy are assessed in t erms of the probability of corr ect guessing. W e defi ne h ( P X Y , ε ) as the maximum probability of correctly guessing Y giv en an auxil i ary random va riable Z , wh ere the maximization is taken over all P Z | Y ensuring that th e pro babilit y of corr ectly guessing X given Z does not exceed ε . W e show that the map ε 7→ h ( P X Y , ε ) is strictly in creasing, concav e, and piecewise linear , which allo ws us to deri ve a close d form expression f or h ( P X Y , ε ) when X and Y are conn ected via a binary-input binary-output channel. F or { ( X i , Y i ) } n i =1 being pairs of in dependent and identically distributed binary random vectors, w e similarly define h n ( P X n Y n , ε ) un der the assumption that Z n is also a binary vector . Th en we obtain a closed f orm expression for h n ( P X n Y n , ε ) for sufficient l y large, b ut nontriv ial va lues of ε . I . I N T RO D U C T I O N A N D P R E L I M I NA R I E S Giv en p riv ate info rmation, repr e sented by a r andom variable X , non-p r iv a te obser vable informa tion, say Y , is generated via a fixed channel P Y | X . Consid er two com - municating agents Alice an d Bob, where Alice o bserves Y and wishes to disclose it to Bob as accu rately as possible in order to recei ve a payoff, but in such a way th at X is kept almost priv a te from him. Gi ven the join t distribution P X Y , Alice chooses a random mapping P Z | Y , a so -called priv acy filter, to gene r ate a ne w rand om variable Z , called the display ed data , such that Bob can guess Y from Z with as small err or pr obability as possible while Z cannot be used to efficiently gue ss X . The tra d eoff betwee n utility and priv acy was add ressed from an info r mation-th eoretic viewpoint in [ 1]–[5], where both utility and pri vacy were measured in terms of informa tio n-theor etic q uantities. In particular, in [2] bo th utility and priv acy were measured in terms of th e mutual informa tio n I . Specifically , the so-called rate-privacy func- tion g ( P X Y , ε ) was defined as the maxim um of I ( Y ; Z ) over all P Z | Y such that I ( X ; Z ) ≤ ε . I n th e mo st stringent privac y setting ε = 0 , called perfect privacy , it was sh own that g ( P X Y , 0) > 0 if a n d o nly if X is weakly independ ent of Y , that is, if the set of vector s { P X | Y ( ·| y ) : y ∈ Y } is linear ly depen d ent. In [4], an equiv alent result was obtain ed in terms of the singular values o f the operato r f 7→ E [ f ( X ) | Y ] . Althou gh a con- nection between th is info rmation- th eoretic priv acy measure and a coding theore m is established in [2] and [6], the use of mu tual inform ation as a priv acy m easure is not satisfactorily m otiv ated in an operationa l sense. T o find a measure of priv acy with a clear operatio nal mean ing, in this p aper we take an estimation -theoretic appr oach and This wo rk was supporte d in part by NSERC of Canada. The authors are with the Department of Mathemati cs and Statis- tics, Queen’ s Uni versit y , Canada. Emails: {asoodehshahab, fady , lin- der}@mast.quee nsu.ca, 13madt@que ensu.ca. define both privac y and utility me asures in terms of the probab ility of guessing correctly . Giv en discrete random variables U ∈ U and V ∈ V , the probab ility of corre ctly gu e ssing U gi ven V is d efined as P c ( U | V ) : = max g Pr( U = g ( V )) = X v ∈ V max u ∈U P U V ( u, v ) , where the first maxim um is taken over all fu nctions g : V → U . It is easy to show that P c satisfies the d a ta processing inequ ality , i.e., P c ( U | W ) ≤ P c ( U | V ) for U , V and W which form the M arkov ch ain U ⊸ − − V ⊸ − − W . Thus, we m e asure p riv acy in ter m s o f P c ( X | Z ) which quantifies the advantage of an adversary ob serving Z in guessing X in a single shot attempt. A similar oper ational m easure of privac y was recen tly propo sed in [7], wh e re P Z | X is said to be ε -priv ate if log P c ( U | Z ) P c ( U ) ≤ ε for all auxiliary rando m variables U satis- fying U ⊸ − − X ⊸ − − Z . T h is requiremen t guar antees that no randomized fu nction of X can be efficiently estimated from Z , which leads to a strong privac y guar a ntee. I n [8], maximal correlatio n [9] was propo sed as anoth er measu re of pr iv a cy . Op erational inter pretations cor r espondin g to this priv acy measure are giv en in [10] fo r the discrete case an d in [11] for a continuo us setup. T o qu antify the co nflict b etween utility and priv acy , we define the privacy-a wa r e guessing function h as h ( P X Y , ε ) : = sup P Z | Y : X ⊸ − − Y ⊸ − − Z, P c ( X | Z ) ≤ ε P c ( Y | Z ) . (1) Due to th e data pr ocessing inequality , we can restrict the priv acy thre shold ε to the interval [ P c ( X ) , P c ( X | Y )] , where P c ( X ) is th e pro bability of corr ectly g uessing X in the absence o f any side info rmation. F or ε close to P c ( X ) , the priv acy gu arantee P c ( X | Z ) ≤ ε intuitiv ely m eans that it is nearly as har d to guess X ob serving Z as it is witho ut observing Z . W e deriv e fu nctional properties of the map ε 7→ h ( P X Y , ε ) . In p articular, we show that it is strictly in- creasing, concav e, and piecewise lin ear . Piecewise linearity (Theor e m 1), which is the most importan t and technic a lly difficult result in th e p a per , allows us to der ive a tight up p er bound on h ( P X Y , ε ) for general P X Y . As a co nsequence of con cavity , we der i ve a closed form expr ession for h ( P X Y , ε ) for any ε ∈ [ P c ( X ) , P c ( X | Y )] w h en X and Y are both bin ary . It is shown (T h eorem 2) that either the Z-chann el or the re verse Z-chan nel achieves h ( P X Y , ε ) in this case dependin g on the backward channel. W e also consider the vector case for a p air of binar y random vectors ( X n , Y n ) unde r an ad ditional constraint that Z n is a b inary random vector . Here, Z n is rev ealed publicly and th e goal is to g uess Y n under th e p riv acy constraint P c ( X n | Z n ) ≤ ε n . This mo del c a n be viewed as a p riv acy-constrained version of the corr elation distil- lation pr oblem studied in [12]. Suppose Alice and Bo b respectively observe Y n and Z n , whe r e { ( Y i , Z i ) } n i =1 is indepen d ent and identically distributed (i.i.d.) accor ding to the joint distribution P Y Z , and assume that they are to design non-co nstant Boolean fun c tions f an d g suc h that Pr( f ( Y n ) = g ( Z n )) is maxim ized. A dimension -free upper boun d fo r this pr obability was giv en in [1 2]. Now suppose P Y Z is not given and Alice is to design P Z | Y (for a fixed Y -m arginal) that maximizes P c ( f ( Y n ) | Z n ) for a given function f wh ile P c ( X n | Z n ) ≤ ε n . W e sho w (Theor e m 3) that if { ( X i , Y i ) } n i =1 is i.i.d. accor ding to P X Y with |X | = |Y | = 2 and P Y | X is a b in ary sym- metric channel, th en the max imum of P c ( Y n | Z n ) u nder the p riv acy constraint P c ( X n | Z n ) ≤ ε n admits a closed form expression for su fficiently large but nontrivial ε . This then provide s a lower bo und for the priv acy-constrained correlation distillation problem due to the trivial fact that P c ( f ( Y n ) | Z n ) ≥ P c ( Y n | Z n ) for any functio n f . W e omit the proo f of m ost of the resu lts due to sp ace limitations. The proofs are a vailable in [13]. I I . S C A L A R C A S E Suppose X and Y are discrete random variables with fi- nite alph abets X = { 1 , . . . , M } and Y = { 1 , . . . , N } , res- pectively , an d with jo int distribution P = { P X Y ( x, y ) , x ∈ X , y ∈ Y } , whose marginals over X and Y are ( p 1 , . . . , p M ) an d ( q 1 , . . . , q N ) , r espectively . Let X re- present th e priv ate data and Y represent a non- priv ate mea- surement of X , which, upon passing it via a priv a cy filter P Z | Y , is publicly displayed as Z . I n order to quantify the conflict between pr i vacy with respect to X and utility with respect to Y , the so-called rate-privac y f u nction g ( P , ε ) was introd uced in [2]. In what follows, we use Arimo to’ s mutual inform ation to gene r alize this definitio n . A. The Utility-P rivacy Function of Order ( ν , µ ) Let H ν ( X ) a n d H A ν ( X | Z ) den ote respectively the Rényi entropy of orde r ν an d Arimoto ’ s condition al entro py of order ν [14], defined for ν > 1 as H ν ( X ) : = 1 1 − ν log X x ∈X P ν X ( x ) ! , and H A ν ( X | Z ) : = ν 1 − ν log   X z ∈Z " X x ∈X P ν X Z ( x, z ) # 1 /ν   . W e define (by contin u ity) H 1 ( X ) = H ( X ) , H A 1 ( X | Z ) = H ( X | Z ) , H ∞ ( X ) = − log P c ( X ) , and H A ∞ ( X | Z ) = − lo g P c ( X | Z ) . Arimoto’s mutua l informatio n of o r der ν ≥ 1 is d e fined as (see, e.g ., [14]) I A ν ( X ; Z ) : = H ν ( X ) − H A ν ( X | Z ) . Thus I A 1 ( X ; Z ) = I ( X ; Z ) . Definition 1. F or a given joint distrib ution P and a pair ( ν, µ ) , ν , µ ∈ [1 , ∞ ] , the utility-priva cy fu nction of or der ( ν, µ ) is g ( ν,µ ) ( P , ε ) : = max P Z | Y ∈ D ν ( P ,ε ) I A µ ( Y ; Z ) , wher e D ν ( P , ε ) : = { P Z | Y : X ⊸ − − Y ⊸ − − Z , I A ν ( X ; Z ) ≤ ε } . Note that D ν ( P , ε ) cannot be empty since all cha n nels P Z | Y with Z indep endent of X satisfy I A ν ( X ; Z ) = 0 , and so th ey belong to D ν ( P , ε ) for a ny ε ≥ 0 . Using a similar technique as in [ 15], one can show that ε 7→ g ( ν,µ ) ( P , ε ) is strictly increasing for any ν, µ ≥ 1 . It is also worth mentionin g that an application of M inkowski’ s inequality implies that th e m ap P Z | Y 7→ exp n ( ν − 1) ν I A ν ( Y ; Z ) o is conv ex fo r ν ≥ 1 , and thus the m a x imum in the definition of g ( ν,µ ) ( P , ε ) is achieved at the bo undar y of the feasible set wh ere I A ν ( X ; Z ) = ε . W e de n ote g ( ∞ , ∞ ) ( P , ε ) and g (1 , 1) ( P , ε ) respectively by g ∞ ( P , ε ) an d g ( P , ε ) . Since I ∞ ( Y ; Z ) = log P c ( Y | Z ) P c ( Y ) , g ∞ ( P , ε ) can be e q uiv alently described as th e smallest Γ ≥ 0 such that P c ( Y | Z ) ≤ P c ( Y )2 Γ , for every P Z | Y satisfying P c ( X | Z ) ≤ P c ( X )2 ε . W e note that for small ε the condition I A ∞ ( X ; Z ) ≤ ε intuitively means that it is n early a s hard f or an adve rsary observing Z to pred ict X as it is without Z . The r efore, g ∞ ( P , 0) quan tifies the efficiency of g uessing Y from Z such that P c ( X | Z ) = P c ( X ) . It is thus in teresting to ob tain a necessary and su fficient condition for P u nder which g ∞ ( P , 0) > 0 . W e obtain such a condition for the spec ial case of binary X and Y in the next section. In general, the map ν 7→ I A ν ( X ; Z ) is not mo n otonic 1 and hen ce P Z | Y might belo ng to D ν ( P , ε ) but n o t to D µ ( P , ε ) for µ < ν . Nevertheless, the following lemma allows u s to obtain upper and lower bo unds for g ( ν,µ ) ( P , · ) in terms of g ∞ ( P , · ) . Lemma 1. Let ( X , Y ) be a pair of random variables having joint distrib ution P an d ν , µ ∈ (1 , ∞ ) . Then g ( ν,µ ) ( P , ε ) ≤ g ∞ ( P , ψ ( ν , ε )) + H µ ( Y ) − H ∞ ( Y ) , wher e ψ ( ν, ε ) : = ν − 1 ν ε + 1 ν H ∞ ( X ) . F urthermor e, we have for ε ≥ H ν ( X ) − H ∞ ( X ) th at g ( ν,µ ) ( P , ε ) ≥ µ µ − 1 g ∞ ( P , ϕ ( ν, ε )) − 1 µ − 1 H ∞ ( Y ) , wher e ϕ ( ν, ε ) : = ε − H ν ( X ) + H ∞ ( X ) . This lemma shows tha t the family of fun ctions g ( ν,µ ) ( P , ε ) for ν, µ > 1 can be bound ed from ab ove and below b y g ∞ ( P , δ ) , wh e re δ depends o n ε and ν . The case ν = µ = 1 is stud ied in [2]. As a result, in the following sectio n we only focus on g ∞ ( P , ε ) . It turns out that it is easier to stud y h ( P , ε ) , defined in (1), instead. It is straightfo rward to verify that g ∞ ( P , ε ) = log h ( P , 2 ε P c ( X )) P c ( Y ) , and hen c e a ll the results f or h ( P , ε ) can be translated to results for g ∞ ( P , ε ) . I n particu lar , per fect priv acy g ∞ ( P , 0) correspo n ds to h ( P , P c ( X )) . Notice th a t h ( P , P c ( X )) > P c ( Y ) is equiv alent to g ∞ ( P , 0) > 0 . As opp osed to I ν ( X ; Z ) with 1 ≤ ν < ∞ , I ∞ ( X ; Z ) = 0 d oes not 1 It is relati vely easy to sho w that if X is uniformly distributed , then I A ν ( X ; Z ) coinci des with Sibson’ s mutual informati on of order ν [14] which is kno wn to be increasing in ν [16, Theorem 4]. Consequentl y , ν 7→ I A ν ( X ; Z ) is increasing ove r (1 , ∞ ] if X is uniformly distrib uted. 0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1 ε h ( ε ) P c ( X | Y ) P c ( X ) Fig. 1. T ypical graph of h ( ε ) . T he dotted line represents the chord connecting ( p, h ( p )) and ( P c ( X | Y ) , 1) which can be viewed as a trivial lower bound for h ( · ) . necessarily imply the independ ence of X and Z (unle ss X is uniformly distributed). In particular, the weak indepen - dence 2 argument fro m [2, Lemma 10] (see also [4]) can n ot be applied f o r g ∞ . For the sake of brevity , we simply wr ite h ( ε ) for h ( P , ε ) when there is no risk of confu sio n . B. Priva c y-A ware Guessing Function It is clear from (1) that P c ( Y ) ≤ h ( ε ) ≤ 1 , and h ( ε ) = 1 if and only if ε ≥ P c ( X | Y ) . A direct application of the Support Lemma [ 17, Lemma 15.4] shows that it is eno ugh to consider ran dom variables Z supp orted on Z = { 1 , . . . , N + 1 } . Th us, th e p riv acy filter P Z | Y can be realized b y an N × ( N + 1 ) stoch astic m atrix F . Let F be the set of all such matrices. Then both u tility U ( P , F ) = P c ( Y | Z ) and pr i vacy P ( P , F ) = P c ( X | Z ) are function s of F ∈ F and we can express h ( ε ) as h ( ε ) = max F ∈F , P ( P ,ε ) ≤ ε U ( P , F ) . It can b e verified that F 7→ P ( P , F ) and F 7→ U ( P , F ) ar e continuo us convex f unctions over F . It can also be shown that the set R : = { ( P ( P , F ) , U ( P , F )) : F ∈ F } is conv ex. Furthermo re, since the g r aph of h ( ε ) is the upper b o undar y of R , we conclude th at ε 7→ h ( ε ) is concave, a nd so it is strictly incr easing a n d co ntinuou s on [ P c ( X ) , P c ( X | Y )] . As a consequen ce, for every ε ∈ [ P c ( X ) , P c ( X | Y )] there exists G such that P ( P , G ) = ε and U ( P , G ) = h ( ε ) . W e call such a priv acy filter G optimal at ε . The f o llowing theorem r ev eals that h ( · ) is a piecewise linear function, as depicted in Fig. 1. Theorem 1. The function h : [ P c ( X ) , P c ( X | Y )] → R is piecewise linear , i.e., ther e exist K ≥ 1 an d thresholds P c ( X ) = ε 0 < ε 1 < . . . < ε K = P c ( X | Y ) such that h is linear on [ ε i − 1 , ε i ] for all 1 ≤ i ≤ K . Consider the map H : F → [0 , 1 ] × [0 , 1] giv en b y H ( P , F ) = ( P ( P , F ) , U ( P , F )) . Let D : = 2 Using a similar proof as in [2], it can be shown that g ( ν,µ ) ( P , 0) > 0 for ν, µ ∈ [1 , ∞ ) if and only if X is weak ly independent of Y . { D ∈ M N × N +1 : k D k = 1 } , w h ere || · || denotes th e E u- clidean norm on M N × ( N +1) , the set of real matrices of size N × ( N + 1) . For G ∈ F define D ( G ) : = { D ∈ D : G + tD ∈ F fo r some t > 0 } . The proo f o f the pr evious theorem is heavily based o n the following techn ica l, yet intuitive, r e su lt: for every G ∈ F , there exists δ > 0 such that H is linear on [ G, G + δ D ] for every D ∈ D ( G ) . The proof techniqu e allows u s to derive the slope of h on [ ε i − 1 , ε i ] , given the family of o p timal filters at a single point ε ∈ [ ε i − 1 , ε i ] . For example, since the family of optim al filters at ε = P c ( X | Y ) is easily obtainable, it is then p ossible to c o mpute h on the last in terval. In the binar y case, this ob servation and th e concavity o f h allow us to show tha t h is line ar on its entire d omain [ P c ( X ) , P c ( X | Y )] . C. B inary Case Assume no w that X and Y are bo th binar y . Let BIBO ( α, β ) den ote a b inary inpu t binary o utput chann el from X to Y with P Y | X ( ·| 0) = ( ¯ α, α ) an d P Y | X ( ·| 1) = ( β , ¯ β ) , where ¯ x : = 1 − x for x ∈ [0 , 1] . Notice that if X ∼ Bernoulli ( p ) with p ∈ [ 1 2 , 1) , then P c ( X ) = p and h ence h ( p ) corre sp onds to th e maxim um o f P c ( Y | Z ) under per fect privac y P c ( X | Z ) = p . Furth ermore, if P Y | X = BIBO ( α, β ) with α, β ∈ [0 , 1 2 ) , then we have P c ( X | Y ) = ma x { ¯ α ¯ p, β p } + ¯ β p. No tice that if ¯ α ¯ p ≤ β p , then P c ( X | Y ) = P c ( X ) = p . The binar y symm etric channel with cro ssover p robabil- ity α , den oted by BSC ( α ) , and also the Z-chan nel with crossover p r obability β , denoted by Z ( β ) , are b oth exam- ples of BIBO ( α, β ) , correspon ding to α = β and α = 0 , respectively . L e t q := Pr( Y = 1) . W e say that p erfect priv acy yields a non-trivial utility if P c ( Y | Z ) > P c ( Y ) for some Z such that P c ( X | Z ) = P c ( X ) , or equivalently , if h ( p ) > ma x { ¯ q , q } . The fo llowing lem ma d etermines h ( p ) in the non-trivial case ¯ α ¯ p > β p . Lemma 2. Let X ∼ Bernoulli ( p ) with p ∈ [ 1 2 , 1) a nd P Y | X = BIBO ( α, β ) with α, β ∈ [0 , 1 2 ) such th at ¯ α ¯ p > β p . Then h ( p ) = ( 1 − ζ q if α ¯ α ¯ p 2 < β ¯ β p 2 , q otherwise , wher e q = α ¯ p + ¯ β p and ζ := ¯ α ¯ p − β p ¯ β p − α ¯ p . (2) Notice that 1 − ζ q > ¯ q if and only if ζ < 1 , which occurs if and only if p ∈ ( 1 2 , 1) . Also, it is straightf orward to show that 1 − ζ q > q if and only if α ¯ α ¯ p 2 < β ¯ β p 2 . I n particular, we h av e th e following n ecessary an d suffi cient condition for non-tr i vial utility un der p erfect p riv acy . Corollary 1. Let X ∼ Bernoulli ( p ) with p ∈ [ 1 2 , 1) a nd P Y | X = BIBO ( α, β ) with α, β ∈ [0 , 1 2 ) such that ¯ α ¯ p > β p . Then g ∞ ( P , 0) > 0 if an d only if α ¯ α ¯ p 2 < β ¯ β p 2 and p ∈ ( 1 2 , 1) . Remark that the condition α ¯ α ¯ p 2 < β ¯ β p 2 can be equ i v- alently written as P X | Y (0 | 1) P X | Y (0 | 0) < P X | Y (1 | 0) P X | Y (1 | 1) . 1 0 1 0 Y X Z β α 1 0 1- ζ ( ε ) ζ ( ε ) (a) α ¯ α ¯ p 2 < β ¯ β p 2 1 0 1 0 Y X Z β α 1 0 1 − ˜ ζ ( ε ) ˜ ζ ( ε ) (b) α ¯ α ¯ p 2 ≥ β ¯ β p 2 Fig. 2. The optimal priv acy filters for P Y | X = BIBO ( α, β ) . The following the o rem establishe s the linear behavior of h when P Y | X = BIBO ( α, β ) . Theorem 2. Let X ∼ Bernoulli ( p ) for p ∈ [ 1 2 , 1) a nd P Y | X = BIBO ( α, β ) with α, β ∈ [0 , 1 2 ) . I f ¯ α ¯ p > β p , then for any ε ∈ [ p, ¯ α ¯ p + ¯ β p ] , we h a ve the follo wing: • If α ¯ α ¯ p 2 < β ¯ β p 2 , then h ( ε ) = 1 − ζ ( ε ) q , wher e q = α ¯ p + ¯ β p a nd ζ ( ε ) := ¯ α ¯ p + ¯ β p − ε ¯ β p − α ¯ p . (3) Furthermore , h ( ε ) is a chieved by the Z- channel Z ( ζ ( ε )) (a s shown in F ig. 2). • If α ¯ α ¯ p 2 ≥ β ¯ β p 2 , then h ( ε ) = 1 − ˜ ζ ( ε ) ¯ q , wher e ˜ ζ ( ε ) := ¯ α ¯ p + ¯ β p − ε ¯ α ¯ p − β p . Mor eover , h ( ε ) is achieved by a r everse Z-chan nel with cr ossover pr obab ility ˜ ζ ( ε ) (a s sho wn in Fig . 2). Pr oo f Sketch. Recall that ε 7→ h ( ε ) is con cav e, and thus its grap h lies above the segment connecting ( p, h ( p )) to ( P c ( X | Y ) , 1) . In particular, h ( ε ) ≥ h ( p ) + ( ε − p )  1 − h ( p ) P c ( X | Y ) − p  . By Lemma 2, the above in equality become s h ( ε ) ≥ h ( p ) + q ( ε − p ) ¯ β p − α ¯ p 1 { α ¯ α ¯ p 2 <β ¯ β p 2 } + ¯ q ( ε − p ) ¯ α ¯ p − β p 1 { α ¯ α ¯ p 2 ≥ β ¯ β p 2 } . (4 ) Since ε 7→ h ( ε ) is piec ewise linear, its right der iv a tive exists at ε = P c ( X | Y ) . Using the g eometric prope rties o f H used to prove Th eorem 1, we can show th at h ′ ( P c ( X | Y )) = q ¯ β p − α ¯ p 1 { α ¯ α ¯ p 2 <β ¯ β p 2 } + ¯ q ¯ α ¯ p − β p 1 { α ¯ α ¯ p 2 ≥ β ¯ β p 2 } , which is equal to the slope of the chord conn ecting ( p, h ( p )) to ( P c ( X | Y ) , 1) described in (4). Th e c o ncavity of h ( · ) thus im plies that the ineq uality (4) is indeed equality . Under the hy p otheses of the previous theorem, f or ev- ery ε ∈ [ P c ( X ) , P c ( X | Y )] th ere exists a Z-ch annel that achieves h ( ε ) . I t can be shown that Z -channe l is the only binary filter with this pr o perty . It is also worth mentionin g that e ven if P Y | X is symmetric (i.e., α = β ), the o ptimal filter cannot be sy m metric, unless X is unifor m, in which case BSC (0 . 5 ζ ( ε )) is also op timal. I I I . I . I . D . B I N A RY S Y M M E T R I C V E C T O R C A S E W e n ext study priv acy aware guessing for a pair of binary rand om vectors ( X n , Y n ) with X n , Y n ∈ { 0 , 1 } n . Recall that in this c a se it is sufficient to consider auxiliary random variables having suppo rts of cardinality 2 n + 1 . Howe ver , this con dition m ay be p ractically in conv enient. Moreover , in the scalar binary case examin ed in the last section we o bserved tha t a binar y Z was sufficient to achieve h ( ε ) . He nce, it is n atural to requir e th e pri- vac y filters to p roduce also binary rando m vectors, i.e., Z n ∈ { 0 , 1 } n , which leads to the fo llowing d efinition. Recall th at the data pr o cessing inequality imp lies that P c ( X n ) ≤ P c ( X n | Z n ) ≤ P c ( X n | Y n ) and hence we ca n assume P c ( X n ) ≤ ε n ≤ P c ( X n | Y n ) . Definition 2. F or a given pair of bina ry ran- dom vectors ( X n , Y n ) , we defin e h n ( ε ) for ε ∈ [ P 1 /n c ( X n ) , P 1 /n c ( X n | Y n )] , as h n ( ε ) : = ma x P 1 /n c ( Y n | Z n ) , (5) wher e the ma x imum is taken over all (not necessarily memoryless) channels P Z n | Y n such tha t Z n ∈ { 0 , 1 } n , X n ⊸ − − Y n ⊸ − − Z n , and P c ( X n | Z n ) ≤ ε n . Note that th is de fin ition does not make any assumption about the p riv acy filters P Z n | Y n except that Z n ∈ { 0 , 1 } n . From an im plementation p oint o f v iew , the simplest p riv acy filter is a memoryless one such that Z k is a noisy version of Y k for k = 1 , . . . , n . More p r ecisely , we are interested in a single BIBO ch annel P Z | Y which, given Y k , generate s Z k accordin g to P Z n | Y n ( z n | y n ) = n Y k =1 P Z | Y ( z k | y k ) . Now , let h i n ( ε ) be defined as max P 1 /n c ( Y n | Z n ) , wh ere the m aximum is taken over such memoryless priv acy filters satisfying P c ( X n | Z n ) ≤ ε n . Let ⊕ d enote mo d 2 ad dition. In wh at follows, we study h n and h i n for th e following setup: a) X 1 , . . . , X n are i.i.d . Bernoulli ( p ) random variables with p ≥ 1 2 , b) Y k = X k ⊕ V k for k = 1 , . . . , n , where V 1 , . . . , V n are i.i.d. Bernoulli ( α ) random variables indepen dent of X n , such that α < 1 2 . W e first deter mine h i n ( ε ) fo r this model and show that (as expected) h i n ( ε ) is in d ependen t of n . Accordin g to this model, P c ( X n ) = p n and P c ( X n | Y n ) = ¯ α n , a nd thu s p ≤ ε ≤ ¯ α . Proposition 1 . I f ( X n , Y n ) satisfies a ) and b ) with p ∈ [ 1 2 , 1) an d α ∈ [0 , 1 2 ) such tha t ¯ α > p , th en h i n ( ε ) = h ( ε ) = 1 − ζ ( ε ) q , for all ε ∈ [ p, ¯ α ] , wher e ζ ( ε ) is give n in (3) an d q = α ¯ p + ¯ αp . Note tha t th e pro position reduce s to T heorem 2 for n = 1 . However , for n ≥ 2 , we have h i n ( ε ) < h n ( ε ) ≤ 11 01 11 01 10 10 00 00 11 10 01 00 1- ζ 2 ( ε ) Fig. 3. The optimal priv acy fi l ter for h 2 ( ε ) for ε ∈ [ ε L , ¯ α ) , where ζ 2 ( ε ) is defined in (6). h ( P X n Y n , ε ) , as implied by the f ollowing theo rem. A channel W is said to b e a 2 n -ary Z-channel, denote d by Z n ( γ ) , if the input and o utput alphabets are { 0 , 1 } n and W ( a | a ) = 1 for a 6 = 1 , W ( 0 | 1 ) = γ , an d W ( 1 | 1 ) = ¯ γ , where 0 = (0 , 0 , . . . , 0 ) and 1 = (1 , 1 , . . . , 1 ) . Theorem 3. Assume that ( X n , Y n ) satisfies a ) and b) with p ∈ [ 1 2 , 1) a nd α ∈ [0 , 1 2 ) such that ¯ α > p . Then, ther e exis ts p ≤ ε L < ¯ α such tha t h n n ( ε ) = 1 − ζ n ( ε ) q n , for ε ∈ [ ε L , ¯ α ] , wh er e q = α ¯ p + ¯ αp a n d ζ n ( ε ) : = ¯ α n − ε n ( ¯ αp ) n − ( α ¯ p ) n . (6) Mor eover , the channel Z n ( ζ n ( ε )) achieves h n ( ε ) in this interval (see F ig. 3 for the case n = 2 ). The mem oryless priv acy filter assumed in h i n ( ε ) is simple to implemen t. Howe ver , it is clea r from Th eorem 3 that this simple filter is not o ptimal even wh en ( X n , Y n ) is i.i.d. since h n ( ε ) is a functio n of n , while h i n ( ε ) is not. The following cor ollary bou n ds the lo ss r esulting fr om using a simple m emoryless filter instead of a n op timal one for ε ∈ [ ε L , ¯ α ] . Clearly , f or n = 1 , there is no gap as h 1 ( ε ) = h i 1 ( ε ) . Corollary 2. Let ( X n , Y n ) satisfy a) and b) with p ∈ [ 1 2 , 1) a nd α ∈ [0 , 1 2 ) such tha t ¯ α > p . I f p > 1 2 and α > 0 , then for ε ∈ [ ε L , ¯ α ] and sufficiently la r ge n h n ( ε ) − h i n ( ε ) ≥ ( ¯ α − ε )[Φ(1) − Φ( n )] , (7) wher e Φ( n ) : = q n ¯ α n − 1 ( ¯ αp ) n − ( α ¯ p ) n . If p = 1 2 , then h i n ( ε ) ≤ h n ( ε ) ≤ h i n ( ε ) + α 2 ¯ α , (8) for every n ≥ 1 and ε ∈ [ ε L , ¯ α ] . Since Φ( n ) ↓ 0 as n → ∞ , (7) implies that, as exp e cted, the g ap between the perfo r mance of the optim al priv acy filter and the optimal me moryless p riv acy filter incr eases as n in c reases. This observation is numerically illustrated in Fig. 4, w h ere h n ( ε ) is plotted as a function of ε for n = 2 and n = 10 . Moreover , (8) implies that when p = 1 2 and α is small, then h n ( ε ) can be ap proxim a te d b y h i n ( ε ) . 0 . 7 0 . 8 0 . 9 1 ε ε L ¯ α Fig. 4. The graphs of h 10 (solid curve), h 2 (dashed curve), and h i (dotted line) gi ven in Theorem 3 and Proposition 1 for i.i.d. ( X n , Y n ) with X ∼ Bernoulli (0 . 6) and P Y | X = BSC (0 . 2) . Thus, we can appr oximate the optimal filter Z n ( ζ n ( ε )) with a simple m emoryless filter given b y Z k = Y k ⊕ W k , wher e W 1 , . . . , W n are i.i.d . Bernoul li (0 . 5 ζ ( ε )) rando m variables that are independ ent of ( X n , Y n ) . R E F E R E N C E S [1] H. Y amamoto, “ A source coding problem for sources with additional outputs to ke ep secret from the rece iv er or wiretappers, ” IEEE T rans. Inf. Theo ry , vol. 29, no. 6, pp. 918–923, Nov . 1983. [2] S. Asoodeh, M. Diaz, F . Alajaji, and T . Linder , “Information ext raction under priv acy constraints, ” Information , vol. 7, 2016. [Online]. A v ailable : http ://www .mdpi.com/2078- 2489/7/1/15 [3] S. Asoodeh, F . Alajaji, and T . L inder , “Notes on informat ion- theoret ic pri vac y , ” in Pr oc. 52nd Annual A llerton Conf. Com, Contr ol, and Computi ng , Sept. 2014, pp. 1272–1278. [4] F . P . Calmon, A. Makhdoumi, and M. Médard, “Fundamenta l limit s of perfec t priv acy , ” in Proc . IEEE Int. Symp. Inf. Theory (ISIT) , 2015, pp. 1796 –1800. [5] A. Makhdoumi, S. Salamatian , N. Fawa z, and M. Médard, “From the information bottleneck to the priv acy funnel, ” in Proc. IEE E Inf. Theory W orkshop (ITW) , 2014, pp. 501–505 . [6] C. T . Li and A. E. Gamal, “Extended Gray-W yner system with complement ary causal side information, ” 2017. [Online]. A vaila ble: arXi v:1701.03207 v1 [7] I. Iss a, S. Kamath, and A. B. W agner , “ An operationa l measure of informati on lea kage, ” in Proc . A nnual Confere nce on Informati on Scienc e and Systems (CISS) , March 2016, pp. 234–239. [8] A. Makhdoumi and N. Faw az, “Pri va cy-uti lity tradeof f under sta- tistic al uncertaint y , ” in Proc . 51st Annual Allerton Conf . Comm, Contr ol, and Computi ng , Oct 2013, pp. 1627–1634 . [9] H. Gebele in, “Das statistische problem der korrelat ion als v ariatio ns- und eigen wert-problem und sein zusammenhang mit der ausgle- ichungsre chnung, ” Zeitsc hrift f ur ange w . Math. und Mech. , no. 21, pp. 364–379, 1941. [10] F . P . Cal mon, M. V aria, M. Méda rd, M. M. Christiansen, K. R. Duf fy , and S. T essaro, “Bounds on inference , ” in Proc. 51st Annual Allerton Conf. Comm, Contr ol, and Computing , Oct 2013, pp. 567– 574. [11] S. Asoodeh, F . Alajaj i, and T . Linder , “Priv acy-a ware MMSE estimati on, ” in Proc. IEEE Int. Symp. Inf. Theory (ISIT) , July 2016, pp. 1989–1993. [12] H. S. W itsenhause n, “On sequence of pairs of dependent random v ariables, ” SIAM Jo urnal on Applied Mathematics , vol . 28, no. 2, pp. 100–113, 1975. [13] S. Asoodeh, M. Diaz, F . Alajaji , and T . Linder , “Priv acy-aw are data guessing and estimat ion, ” T o be submitted, dra ft available at http:// www . m ast.queensu.ca/ ~shahab/ ISIT_Proof s.pdf . [14] S. V erdú, “ α -mutual information, ” in Proc . Inf. T heory and Appli- cations W orkshop (IT A), 2015 , Feb . 2015, pp. 1–6. [15] H. W itsenhausen and A. W yner , “ A conditi onal entropy bound for a pair of discrete random varia bles, ” IEEE T rans. Inf. Theory , vol. 21, no. 5, pp. 493–501, Sep. 1975. [16] S. W . Ho and S . V erdú, “Con vexit y/conca vity of Rény i entrop y and α -mutual information, ” in Pr oc. IEE E Int. Symp. Inf. Theory (ISIT) , June 2015, pp. 745–749. [17] I. Csisz ár and J. Körner , Informati on Theory: Coding Theorems for Discr ete Memory less Systems . Cambridge Uni versity Press, 2011.