The Category-Theoretic Arithmetic of Information

The Category-Theoretic Arithmetic of Information Benjami n Allen Boston Universit y Department of Mathemat ics and Stat i stics 111 C umming ton St Boston, MA 02 1 15 Octob er 26 , 2018 Abstract W e highligh t the underlying catego ry-theoretic structure of mea- sures of information flo w. W e present an axiomati c f ramew ork in whic h comm unication systems are rep r esen ted as morphisms , and in- formation flow is c haracterized b y its b eha vior when comm unication systems are combined. Our f r amew ork includes a v ariet y of discrete, con tinuous, and, conjecturally , quantum information measures. It also includes some familiar mathematic al constructs not normally associ- ated w ith in formation, su c h as v ector space dimension. W e discuss these examples and pro ve basic results from the axioms. 1 In tro du ction Information theory characterizes the transmission of informatio n using a v ari- et y of measures, including discrete and contin uous v ersions of en tropy , m utual information, and c hannel capacit y . Quantum information theory has added to this list b y intro ducing notions of quan t um c hannel capacit y . In this w ork w e iden tify a category-theoretic structure underlying these measures. W e show that communic ation systems ha v e a natur a l represen- tation as morphisms, and us ing this repres en tatio n, we give axioms that 1 c haracterize information flow. Unlik e other axiomatizatio ns of informat ion [11, 5 , 7, 1, 13], w e do not require specific mathematical data suc h as a ran- dom v ariable or set par t it ion. O ur framew ork thereb y encompasses not only a single inf o rmation m easure or family of measures, but a wide v ariety of func- tions used in discrete, con t inuous, and, conjecturally , quan tum information theory . V ector space dimension also satisfies the axioms of an informatio n measure, giving a dditional supp ort to connections b etw een informatio n and dimension that ha v e b een not iced elsewhere [9, 3]. W e hop e our w ork will help unify and extend the r esults of information theory b y stimu lating the disco v ery of general theorems and new w ays to measure information The or g anization of this pap er is as follo ws: In section 2 w e in tro duce the represen tation of comm unicatio n systems as mor phisms, and in section 3 we discuss how suc h systems m a y b e for ma lly comb ined using category-theoretic op erations. In section 4 w e presen t our axiomatic framew ork describing the b eha vior of information under these o p erations. Section 5 explores the v ar- ious settings to whic h our framew ork applies, and in section 6 w e use the axioms t o prov e basic results a b out the general prop erties of information. 2 Comm unicati on Syst ems W e view information as something transmitted b y a comm unication system . A comm unication system consists of a source and a destination, together with a metho d o f transmitting information bet w een them in the form of mes sages. This picture can be s een a s a simplified v ersion of Shannon’s mo del, depic ted in figure 1. W e us e the term “comm unication system” v ery generally . A p erson look- ing at a picture is an example of suc h a system, since information is trans- mitted from the picture to the p erson. Our fo cus here is on systems with mathematical represen tations, but the framew ork w e presen t may also giv e insigh t in to less rigorous situations. Comm unication systems hav e a natural represen tation as category-theoretic morphisms. They are directed relations b et w een tw o ob jects satisfying the three basic prop erties of morphisms: a) t w o comm unication systems can b e “comp osed” by passing a message through one and then the other, b) this comp osition is asso ciativ e, and c) “identit y” comm unication systems exist, wherein the source is t he same a s the destination and the message stays as is. W e therefore mo del comm unication systems as morphisms in v arious 2 Figure 1: Shannon’s mo del of a comm uncatio n system [1 1]. categories In most of the categor ies w e consider here, the ob jects a re sets w ith p erhaps some additional structure, and the morphisms are set mappings pre- serving the structure. The elemen ts of the sets represen t p ossible messages, and the morphisms ma p messages sen t onto messages receiv ed. Informatio n can then b e quan tified in terms of ho w m uc h the receiv ed message tells the receiv er ab out whic h message w a s sent. W e note, how ev er, that our formal- ism ma y a lso b e used with m ore abstract categories (see section 5.5) , whe rein the in terpretation of morphisms as comm unication systems is less in tuitiv e. 3 Com bin ing Morphis ms W e will c haracterize information in terms of its b eha vior when comm unica- tion systems a r e com bined. How ev er, there are sev eral meaningful wa ys to com bine comm unication syste ms, eac h o f whic h cor r esp o nds to an op eration on morphisms. One suc h op eration is comp osition, as describ ed abov e. In this sec tion w e define t wo new op erations on mor phisms and in terpret them in terms of comm unication systems. Let C b e a category . F or morphisms f , g ∈ hom( C ), we define the external pr o duct f ˆ × g ∈ hom ( C ) to b e the pro duct, if it exists, of f and g in the category ˆ C whose ob jects a re morphisms in C and whose morphism s h 1 → h 2 3 are commutativ e diagrams in C of the form C 1 h 1   / / C 2 h 2   D 1 / / D 2 . If no suc h pro duct exists, f ˆ × g is undefined. Ot herwise, f ˆ × g is w ell-defined up to isomorphism in ˆ C . The second op eration is only defined for morphisms with the same do - main. Let morphisms f , g hav e domain A . W e define the internal pr o duct f × A g ∈ hom ( C ) to b e the pro duct, if it exists, of f and g in the category C A whose ob jects are morphisms with domain A , and w hose morphisms h 1 → h 2 are commutativ e diagrams in C of the form A h 1   h 2 ! ! C C C C C C C C C 1 / / C 2 . As ab ov e, if no suc h pro duct exists, f × A g is undefined; otherwise, it is w ell-defined up to isomorphism in C A . T o unders tand these op erations in terms of comm unication systems, con- sider the category FinSet of finite sets and finite set mappings, where eac h morphism is inte rpreted as a corresp ondence betw een sen t and receiv ed mes- sages. If f and g are set mappings with domain S , the internal pro duct f × S g is the function s 7→  f ( s ) , g ( s )  ev aluating f and g on the same e lemen t of S . This corresp onds to sending the same mes sage through t w o differen t systems sim ultaneously . f ˆ × g , on the other hand, is the map ( s 1 , s 2 ) 7→  f ( s 1 ) , g ( s 2 )  taking tw o inputs and ev aluating f on the first a nd g on the second. This corrp esp onds to sending t w o differen t messages through differen t systems. Somewhat mo r e generally , suppose the pro duct of an y t wo ob jects exists in o ur category C . Then the external and internal pro ducts alw ays exist and are, up to isomorphism, the unique morphisms making the follo wing diagrams comm ute: A f   A × B o o f ˆ × g   / / B g   X X × Y o o / / Y 4 for external pro ducts and A f × A g   f { { v v v v v v v v v v g # # G G G G G G G G G G X X × Y o o / / Y for in ternal pro ducts. 4 Axioms for Information F unction s W e now presen t axioms c hara cterizing the b ehavior of informat io n measures. These axioms w ere chose n b ecause a) they represen t prop erties of informa- tion we b elieve to b e fundamental, and b) they app ear to b e a minimal set necessary to establish basic lemmas (see section 6.) F or a category C whose morphisms represen t comm unication systems , w e define an information function to b e a function that quantifie s the amo unt of information receiv ed b y the destination as a message is sen t thro ugh a system. Mathematically , an information function I assigns to eac h mor phism f of C a nonnegat ive real num b er I ( f ), sub ject to the following axioms. First, if t w o commun ication systems are mathematically iden tical, the information flo w through eac h is the same. Axiom 1. Invarianc e: If mo r phisms f and g are isomorphic in ˆ C , then I ( f ) = I ( g ) . Second, if t wo different messages are sen t through differen t sys tems, so that the receiv ed messages are indep enden t of eac h other, the amoun t of information in b oth t hem is the sum of the information in each of them. Axiom 2. External A dditivity: I f f ˆ × g exists , I ( f ˆ × g ) = I ( f ) + I ( g ). If the same mes sage is sen t through differen t systems sim ultaneously , the total info r ma t io n g a in cannot exceed t he sum of the information obta ined through eac h system, and ma y b e strictly less due to redundancies in the receiv ed messages. More strongly , w e require: Axiom 3. Internal Str ong Sub additivity: If f , g , and h hav e the same doma in A , then I ( f × A g × A h ) ≤ I ( f × A g ) + I ( g × A h ) − I ( g ) , if the ab ov e pro ducts exist. 5 That is, the information gained from the tripartite system f × A g × A h cannot exc eed the amoun t gained from f × A g , plus that gained f r o m g × A h , min us the amoun t I ( g ) that is redundan t to f × A g and g × A h . The four t h axiom concerns the case where a single message is sen t through t wo systems in sequence ; i.e. the message receiv ed from the first system is rela yed through the second. No new information can b e gained from passing the mess age through the second system; this is kno wn as the data pr o c essing ine quality . F urthermore, if no information is lo st in sending the message through the second system, then it is p ossible to reconstruct the first receiv ed message from the second. Axiom 4. Monotonicity: Given a diagram A f / / B g / / C , I ( g ◦ f ) ≤ I ( f ) with equalit y if a nd only if there is a morphism s : C → B suc h that s ◦ g ◦ f = f . Finally , the amount o f infor mation that can b e sen t thr o ugh a system is limited the rang e of messages w hic h can be rec eiv ed. One cannot impro v e on a system in whic h t he sent messages matc h the receiv ed messages p erfectly . Axiom 5. D estination Matching: F or any morphism f with codomain B , I ( f ) ≤ I (id B ). 5 Examples of Information F unctions W e now explore mathematical informa t io n functions encompassed b y our framew ork, including m an y clas sical information measures as w ell a s familiar constructs not usually seen as related to information. W e also sho w how our fram work migh t b e applied to quan tum c hannel capacities. With the exception of these quan t um capacities, satisfaction o f the axioms is either a w ell-know n result or a simple exercise. 5.1 Discrete Comm unication In the simplest mathematical mo dels o f comm unication, there is a finite set of p ossible messages to send and a deterministic mapping from messages sen t to messages receiv ed. Such systems can b e represen ted b y morphisms in the category FinSet of finite sets and set mappings. 6 5.1.1 Hartley E n tr op y The info r ma t io n flo w through a discrete system can be quan tified in terms o f the n um b er of p ossible receiv ed messages. The greater v ariet y in the mess- sages whic h can be rece iv ed, the more information is gained upon rec eiving a particular message. One simple w ay to measure this v ariety is the follo wing: for a set mapping f : A → B , we define the Hartley entr opy of f to b e H 0 ( f ) = log | f ( A ) | , the logarithm (customarily ta k en ba se 2) of the cardinality of f ’s image. Hartley entrop y measures the exp ected n um b er of bits needed to enco de a receiv ed message if all po ssible receiv ed messages are equally like ly . 5.1.2 Shannon En trop y The Shannon en tropy mo difies H 0 to incorp orate unequal pro ba bilit ies of receiv ed messages. F or a mapping f : A → B of finite sets, w e define the Shannon entr opy of f to b e H ( f ) = − X b ∈ B | f − 1 ( b ) | | A | log | f − 1 ( b ) | | A | . If all sen t mes sages a are equally lik ely , H ( f ) quan tifies the uncertain t y of the receiv ed message as the expected n umber o f bits needed to enco de f ( a ) using an optima l enco ding. By a theorem of Aczel, F orte, and Ng [2], the only inf o rmation f unctions on the category Fin Set are linear com binations of the Sh annon and Hartley en tropies. 5.1.3 Noisy Information The categor y FinSet has limitations as a s etting for mo delling dis crete com- m unication systems. There is no w a y to r epresen t comm unication errors induced b y noise, or to record the fa ct that some me ssages are more lik ely to b e sen t t ha n others. W e can o vercome b oth of these limitations b y consid- ering our ob jects to hav e tw o parts: a visible par t represen t ing the messages that can be sen t or receiv ed, and a hidden par t represen t ing noise that ma y affect transmission. 7 Let NoisyFinSet b e the category whose ob jects are pairs ( M , A ) of finite sets, together with a surjectiv e set map π A : M → A . F or eac h a ∈ A , the preimage M a ≡ π − 1 A ( a ) ⊂ M represen ts the en vironmen tal noise fa ctors that migh t b e sen t along with a . Morphisms in this category are diagrams M f / / π A   N π B   A B . F or eac h intend ed message a ∈ A , the actual transmitted data is an elemen t of the preimage M a , represen ting b oth the original message and the no ise. Differen t elemen t s in this preimage may map to differen t elemen ts of B under π B ◦ f , in a ccordance with the po ssibilit y that noise may c hange the rec eiv ed message. The map π B ma y be used to mo del the deco ding of an error- correcting co de. F or suc h a morphism w e de fine the noisy Shannon information in the fol- lo wing manner: Consider M as a proba bility space with normalize d coun ting measure (that is, consider eac h elemen t of M as equally lik ely to o ccur.) Then π A and π B ◦ f can b e in terpreted as random v ar ia bles with v alues in A and B respectiv ely . Define N I ( f ) = I ( π B ◦ f ; π A ), the mutual information of t hese tw o v ariables. This quan tit y ma y b e computed precisely as N I ( f ) = 1 | M | X a ∈ A b ∈ B | M a ∩ M b | log | M a ∩ M b | | M b | − 2 | A | lo g | A | , where M b is the preimage f − 1 ( N b ) = f − 1 ( π − 1 B ( b )) ⊂ M , i.e. the set of all noise elemen ts m ∈ M that cause message b to b e receiv ed. This function generalizes the Shannon entrop y as defined a b o v e. If an elemen t m ∈ M is c hosen randomly with uniform probability , then N I ( f ) equals the a ve rage amoun t o f Shannon information that the receiv ed message π B ◦ f ( m ) ∈ B imparts ab out the sen t message π A ( m ). This formalism also a llo ws for un- equal probabilit y in sen t mess ages: the probability of a message a ∈ A b eing sen t is prop ortio nal to the cardinalit y of the preimage M a . 5.1.4 Channel Capacit y Channel capacity is the maxim um a moun t of information that can b e sen t through a noisy system, where the maxim um is take n o v er all probability 8 distributions on the set of sen t messages. T o represen t channel capacit y a s an info rmation function, w e again use the categrory NoisyFinSet . Consider a morphism M f / / π A   N π B   A B . F or an y message a ∈ A w e consider eac h elemen t of the preimage π − 1 A ( a ) to b e equally lik ely . Using this rule, giv en any proba bility distribution p A on A , we can asso ciat e a probability distribution o n M , and, via π B ◦ f , a join t distribution p on A × B and a marginal distribution p B on B . W e define the channel c ap acity C ( f ) to b e the maxim um o v er all probabilit y distributions on A of the m utual information betw een the sen t message a ∈ A and the receiv ed message b ∈ B : C ( f ) = max p A X a ∈ A b ∈ B p ( a, b ) log p ( a, b ) p A ( a ) p B ( b ) . 5.2 Con tin uous Comm unication Man y real-w orld commu nication systems hav e a con tinuous range of mes- sages that c an be sen t or receiv ed. T o mo del these systems w e use the category P rob whose ob j ects are pro babilit y spaces, i.e. measure spaces ( M , µ ) with µ ( M ) = 1. W e define the morphisms in Prob t o b e measurable functions f : ( M , µ ) → ( N , ν ) that are b ackwar ds me asur e pr eservin g , i.e. µ ( f − 1 ( U )) = ν ( U ) for each measurable U ⊂ N . This condition guaran tees that t he probability measure induced b y f on N agrees with ν . In ternal pro ducts of morphisms are not guarantee d to exist in this cat- egory . The ob vious candidate, x 7→ ( f ( x ) , g ( x )), do es not in general satisfy the bac kw ards measure pr eserving condition. Ex ternal pro ducts, how ev er, do exist: for f : ( M 1 , µ 1 ) → ( M 2 , µ 2 ) and g : ( N 1 , ν 1 ) → ( N 2 , ν 2 ), the external pro duct f ˆ × g is the natural map b et we en the pro duct spaces M 1 × N 1 and M 2 × N 2 , sending ( x, y ) to ( f ( x ) , g ( y )). It is easily v erified t ha t this map is bac kw ards measure preserving. 9 5.2.1 Noisy Information T o represen t imp erfect commun ication, w e add “noise” t o this category in the same manner as for FinSet , by forming a category NoisyPr ob whose ob jects are pairs  ( M , µ ) , ( A, α )  with morphisms π A : M → A . Morphisms in NoisyProb are again giv en b y comm utat ive diag rams ( M , µ ) f / / π A   ( N , ν ) π B   ( A, α ) ( B , β ) . Suc h a morphism f induces a surjectiv e measurable map ˜ f : M → A × B sending m ∈ M to ( π A ( m ) , π B ◦ f ( m )). ˜ f and the measure µ on M induce a probabilit y measure ρ on A × B . If ρ is absolutely con t inuous with resp ect to µ × ν , we define the (c ontinuous) noisy Sha nnon information of f to b e N I cont ( f ) = Z A × B ln dρ d ( µ × ν ) dρ, where dρ d ( µ × ν ) is the Radon- Nik o dym deriv ativ e of ρ with resp ect to µ × ν . If ρ is not absolutely con tinuous with resp ect to µ × ν , we leav e I ( f ) undefined. Our definition agrees with the usual form ula for con t inuous mutual infor ma t io n I = Z A × B p ( x, y ) ln p ( x, y ) p A ( x ) p B ( y ) d ( µ × ν ) under the substitutions p ( x, y ) = dρ d ( µ × ν ) and p A ( x ) ≡ p B ( y ) ≡ 1 with resp ect to the probability measures α and β , respectiv ely . (It may come as a surprise that N I cont for a noi s e less system, wherein π A is an isomorphism, is usually undefined if A con tains uncountably man y p oin ts. This is b ecause a comm unication system that can faithfully transmit an y of uncoun tably man y inputs can send any finite amount of informatio n in a single message.) 5.2.2 Channel Capacit y Channel capacit y is obtained b y maximizing the amoun t of transmitted in- formation ov er the set of a ll probabilit y distributions on the space of sen t 10 messages. W e represen t such probabilit y distributions b y measurable func- tions p : A → R ≥ 0 with R A p dα = 1. F or a particular message a ∈ A , we view all en vironmen tal noise factors in the preimage π − 1 A ( a ) as equally likley; w e therefore define a new probability measure µ p on M by µ p ( U ) = Z U p ◦ π A dµ for eac h measurable U ⊂ M . Giv en a morphism f as ab ov e, the map ˜ f : M → A × B induces a probabilit y measure ρ p on A × B . The c ontinuous ch annel c ap acity of f is defined as C cont ( f ) = sup p Z A × B ln dρ p d ( µ × ν ) dρ p , if suc h a suprem um exists; otherwise C ( f ) is undefined. 5.3 Quan tum Comm un ication F or systems that comm unicate using pro cesses gov erned by quan tum me- c hanics, the transmission of infor mation is describ ed by quantum informa- tion theory . Quan tum information theory is a young field with many op en problems; in particular, w e cannot sa y a t this time whether the v arious ca- pacities of a quan tum c hannel satisfy our axioms for information functions. Ho wev er, our category-theoretic framew ork s uggests new pro blems and casts existing ones in a new ligh t. Consider a finite-dimen sional quan tum system represen ted by Hilb ert space H . A “mess age” in this system is represen ted by a densit y mat rix: a p o sitiv e-semidefinite Hermitian op erator of trace one acting on H . Let D ( H ) denote t he space of all densit y matrices ov er the Hilb ert space H . Giv en tw o s uc h spaces D ( H 1 ) and D ( H 2 ), the na t ur a l c hoice for a morphism b et w een them is a completely p ositiv e trace-preserving linear map. W e call suc h maps CP m a ps or quantum channe ls . W e denote by Quan t the category whose o b jects a re spaces D ( H ) of densit y matrices ov er finite-dimensional Hilb ert spaces, a nd whose morphisms are CP maps. External pro ducts in this category are give n by tensor pro duct of CP maps. Internal pro ducts do not exist in all cases; for instance, t he internal pro duct of the iden tity map ρ 7→ ρ with itself would hav e to b e ρ 7→ ρ ⊗ ρ , but this is nonlinear and violates the no-cloning theorem. How ev er, there 11 are imp o r tan t cases where in t ernal pro ducts do exist; for example, giv en a bipartite system AB , the in ternal pro duct of the partial tra ces T r A : D ( A ⊗ B ) → D ( A ) and T r B : D ( A ⊗ B ) → D ( A ) is the iden tity map D ( A ⊗ B ) → D ( A ⊗ B ). There are sev eral different notions of capacit y for a quantum c hannel, dep ending on the in tended application. The quantum c ap acity measures the capacit y of the c hannel to send quan tum states intact. The classic al c ap ac- ity measures the capacit y to send classic al inf ormation. The entanglemen t- assiste d classic al c ap acity is the capacit y to send classical information if the sender and receiv er are allo w ed to share an a rbitrary n umber of en tangled quan tum states prior to transmission. F or mathematical definitions of these quan tities we refer our readers t o the literature [10, 4]. All the abov e capacities satisfy in v ariance, monoto nicit y , and destination matc hing. External additivit y for these capacities is a famo us op en prob- lem; it amoun ts to the question of whe ther quan tum en ta ngled mess ages can b e comm unicated more efficien tly than unen tangled messages. F or partial results on this problem, see [6, 12]. I n ternal strong subadditivit y of these capacities has no t, to our kno wledge, b een studied. This question is lik ely related to the strong subadditivit y of V on Neumann en trop y , prov en b y Lieb and Rusk ai [8]. 5.4 V ector Comm unication There are strong reasons to suspect a relationship b etw een informatio n and dimension. F o r example, the dimension of a v ector space equals the nu m b er of co ordinates required to sp ecify a p oint in the space, m uc h lik e t he Shannon en tropy equals t he av erage n um b er of bits needed to sp ecify t he v alue of a random v ariable. Deep connections ha v e also b een found b etw een notions of dimension a nd complex it y [9, 3]. It is therefore not surprising that v ector space dimension is also an information function. Supp ose w e ha ve a comm unication system in whic h the set of po ssible messages is represen t ed by a finite-dimensional v ector space, and sen t mes- sages are mapp ed linearly onto receiv ed messages. W e represen t s uc h systems using the catego ry K - FinV ect of finite-dimensional ve ctor spaces ov er a field K , with morphisms giv en b y linear ma ps. F or a morphism f : V → W , w e define d ( f ) = dim f ( V ) . 12 d ( f ) can b e inte rpreted as the amo unt b y whic h kno wledge of f ( v ) ∈ W reduces t he dimensional uncertain t y of v ∈ V . 5.5 Information F unctions on Du al Categories Finally , w e presen t a pair of infor mation functions whose in terpretation in terms of comm unication systems is not clear. The categories for these are the duals of the familiar categories FinS et and K - FinV ect . In a dual category , the in ternal and external pro duct op eratio ns corresp ond to similarly defined in ternal and external c opr o duct operations in the original catego r y . 5.5.1 Cardinalit y of I mage Consider the category FinSet ∗ whose ob jects are finite sets and whose mor- phisms f ∗ : A → B cor r espo nd to set mappings f : B → A . The external pro duct of f ∗ : A → B and g ∗ : C → D is the morphism f ∗ ˆ × g ∗ : A ⊔ C → B ⊔ D ( ⊔ denotes disjoint union) corresp onding to the set mapping f ˆ + g : B ⊔ D → A ⊔ C f ˆ + g : x 7→ ( f ( x ) if x ∈ B g ( x ) if x ∈ D . The in ternal pro duct of f ∗ : A → B a nd g ∗ : A → C is the morphism f ∗ ˆ × g ∗ : A → B ⊔ C corresp onding to the set mapping f + A g : B ⊔ C → A f + A g : x 7→ ( f ( x ) if x ∈ B g ( x ) if x ∈ C . The function mapping f ∗ to | f ( B ) | , the cardinality of the image of the corresp onding set map f , is an information function in this category . It is uncle ar, how ev er, in what sense the dual o f a set mapping represen ts a comm unication system. 5.5.2 Dimension of Image Similarly , w e can explore the category K - FinV ect ∗ whose ob jects are finite dimensional v ector spaces and whose morphisms f ∗ : V → W corresp ond 13 to linear maps f : W → V . The external pro duct of f ∗ : V → W a nd g ∗ : T → U is the morphism f ∗ ˆ × g ∗ : V ⊕ T → W ⊕ U corresp onding to the linear map f ˆ + g : W ⊕ U → V ⊕ T f ˆ + g : w ⊕ u 7→ f ( w ) ⊕ g ( u ) for w ∈ W , u ∈ U . The internal pro duct of o f f ∗ : V → W and g ∗ : V → U is the mo r phism f ∗ ˆ × g ∗ : V → W ⊕ U corresp onding to the linear map f + V g : W ⊕ U → V f + V g : w ⊕ u 7→ f ( w ) + g ( u ) for w ∈ W, u ∈ U . F or a morphism f ∗ : V → W , the function I ( f ∗ ) = dim( f ( W )) is an information f unction in this category . Since dimension of image is also a information function in the original cageory K - FinV ect , this function migh t b e called a bi-information function , in that it is an information function on a category and its dual. The existence of a bi-information function fo r K - FinV ect is doubtless related to the existence of bipro ducts in this category . 6 Basic Results W e no w establish some basic facts ab out information functions. W e start b y sho wing that pro ducts and compositions are w ell-defined o n isomorphis m classes in ˆ C a nd C A . Prop osition 1. Supp ose a ∼ = b and c ∼ = d in ˆ C , and e ∼ = f and g ∼ = h in C A . Then, assumin g the fol lo w ing c omp ositions and pr o ducts exist, (a) a ◦ c ∼ = b ◦ d in ˆ C . (b) a ˆ × c ∼ = b ˆ × d in ˆ C . (c) a ◦ e ∼ = b ◦ f in C A . (d) e × A g ∼ = f × A h in C A . Pr o of. Exercise. 14 No w let I b e an arbitrary information function. The fo llo wing prop osition sa ys that no comm unication sys tem can do b etter than one whic h repro duces the inputs exactly . Prop osition 2. Source match ing: F or any morphism f with domain A , I ( f ) ≤ I (id A ) . Pr o of. I ( f ) = I ( f ◦ id A ) ≤ I (id A ) by monoto nicit y . Prop osition 3. Monotonicit y with respect to internal pro ducts: L et f and g b e morphisms with domain A . Assuming the r elevant pr o ducts exist, (a) I ( f × A g ) ≥ I ( f ) (b) I ( f × A f ) = I ( f ) . In other w ords, y ou will nev er lose an y information b y sending the same message sim ultaneously through tw o differen t systems vers us sending it through just one of them, but if the t w o sys tems are iden tical, y ou w on’t gain any information either. Pr o of. (a) By the definition of pro ducts there is a morphism π 1 : f × A g → f in C A , whic h corresp onds to a diagra m A f × A g   f A A A A A A A A C / / B . The result now follows from monotonicit y . (b) W e m ust n o w sho w I ( f × A f ) ≤ I ( f ). Again b y the definition of pro ducts, there is a unique morphism δ : f → f × f in C A making the follo wing diagram commute: f δ   id | | x x x x x x x x x x id # # F F F F F F F F F F f f × A f o o / / f . 15 The morphism δ represen ts a diag ram A f   f × A f @ @ @ @ @ @ @ B / / C , from which the result follow s b y monotonicit y . F urther results dep end on the existence of terminal ob jects in o ur cate- gory C . A o b ject T is terminal if there is exactly one morphism in to T fro m an y o b ject A . An y singleton set is a terminal o b ject in FinSet . In tuitiv ely , a destination represen ted b y a terminal ob ject is unable to discriminate b e- t wee n messages, and th us cannot receiv e information. W e prov e this and other results in the follo wing prop ositions: Prop osition 4. L et T b e a terminal element of C . F or a fixe d obje ct A , let t b e the u nique morphism t : A → T . The n, assuming t he appr opriate pr o ducts exist, (a) id T and t ar e terminal ele m ents in ˆ C and C A , r esp e ctively. (b) F or any morphis m g ∈ hom C , g ˆ × id T ∼ = g in ˆ C . If g h as domai n A , then g × A t ∼ = g in C A . (c) L et B b e an obje ct for w h ich A × B exists. Then π B ∼ = t ˆ × id B in C A × B . Pr o of. (a) Exercise. (b) Combine part (a) with the fact that A × T ∼ = A for any ob j ect A , in an y category with terminal o b ject T . (c) Let s and t b e the unique morphisms t : A → T , s : B → T . Let e b e the unique morphism e : B → T × B making the follo wing diagram comm ute: B s { { w w w w w w w w w e   id B # # G G G G G G G G G T T × B o o / / B 16 Then the diagrams A × B t ◦ π A { { x x x x x x x x x e ◦ π B   π B # # G G G G G G G G G T T × B o o / / B and A × B t ◦ π A { { x x x x x x x x x t ˆ × id B   π B # # G G G G G G G G G T T × B o o / / B also comm ute. By the definition o f pro ducts w e m ust hav e e ◦ π B = t ˆ × id B , and the result no w follows from the fact that e is an isomor- phism. Corollary 5. L et f : A → C b e any morphism , and let e and t b e as ab ove. Then (a) I ( f ˆ × id T ) = I ( f × A t ) = I ( f ) for any morphi sm f , whe never these pr o ducts ar e defi ne d. (b) I (id T ) = I ( t ) = 0 . (c) Subadditivity: I ( f × A g ) ≤ I ( f ) + I ( g ) for any two morphisms f , g with domain A . (d) I f the pr o duct A × B exists for an ob j e c t B , then I ( f ◦ π A ) = I ( f ) . P art (d) sa ys that the information flo w through a comm unication system f is not a ff ected by the presence of “irrelev an t” information in B . Pr o of. P arts (a) follows direcly fro m the prop osition. F or part (b), I (id T ) = 0 follo ws from par t (a) and additivity , then I ( t ) = 0 b y destination matc hing. P art (c) is prov en b y subsituting t into strong subadditivit y , I ( f × A t × A g ) + I ( t ) ≤ I ( f × A t ) + I ( t × A g ) , and in v oking previous results. F or part (d), I ( π A ) = I (id A ) b y Prop osition 4(c), and the result follow s from Prop osition 1(a) . 17 7 Conclus ion Mathematical abstractions like groups and topolog ical spaces hav e the p o w er to illuminate connections b et w een differen t ob jects of study and inspire the disco v ery of new ob jects. W e hop e this presen t abstraction will lead to the disco v ery of ne w w ays to measure information, and deep en understanding of information’s g eneral prop erties. In the future, w e aim to prov e deep er results and inv estigate information functions relev an t to sp ecific applied situations. References [1] S. Abe. Axioms and unique ness theorem for Tsallis en trop y. Physics L etters A , 271(1 - 2):74–79 , 20 0 0. [2] J. Aczel, B. F orte, and CT Ng. Wh y the Shannon and Ha rtley En tropies Are “Natural”. A dvanc es in Applie d Pr ob ability , 6(1):131–1 46, 1974. [3] J.M. Hitc hco c k. Effe ctive fr actal dim ension: foundations and appli c a- tions . PhD thesis, Io w a State Univ ersit y , 2003. [4] M. Keyl. F undamen t als of quantum information theory. Ph ysics r ep orts , 369(5):431 –548, 2002. [5] A.I.A. Khinc hin. Mathematic al F ounda tion s of Information The ory . Do v er Publications, 19 57. [6] C. King. Additivit y f o r unital qubit c hannels. Journal of Mathematic al Physics , 43:4641, 2 002. [7] PM Lee. On the axioms of information theory. Ann. Math. S tatist , 35:415–41 8, 1964. [8] E.H. Lieb and M.B. Rusk ai. Pro of of the strong s ubadditivit y of quan t um- mec hanical en trop y . Journal of Mathematic al Physics , 14:1938–1 941, 197 3. [9] J.H. Lutz. D imension in complexit y classes. SIAM Journal on Comput- ing , 3 2(5):1236– 1259, 2003. [10] M.A. Nielse n and I.L. Ch uang. Quantum Computation and Quantum Information . Cam bridge Univ ersit y Press, 200 0. 18 [11] C. Shannon. A mathematical theory o f comm unication. Bel l Systems T e chn o lo gic al Journal , 27 :379–423, 1948. [12] P .W. Shor. Equiv alence of Additivit y Questions in Q ua n tum Infor ma- tion Theory . Communic ations in Mathem a tic al Physics , 246(3):4 53–472, 2004. [13] H. Suy ari. Generalization of Shannon- Khinc hin axioms to no nexten- siv e systems and the uniqueness theorem for the nonextensiv e en tro py . Information T he ory, IEEE T r ansactions on , 50(8):1783 –1787, 2004. 19