General Algorithms for Testing the Ambiguity of Finite Automata

General Algorithms f or T esting the Ambiguity of Finite A utomata Cyril Allauzen 1 , ⋆ , Mehry ar Mohri 1 , 2 , and Ashish Rastogi 1 1 Courant Institute of Mathematical Sciences, 251 Mercer Street, New Y ork, NY 10012. 2 Google Research, 76 Ninth A venu e, Ne w Y ork, NY 10011. Abstract. This paper presen ts efficient algorithms for testing the fi nite, poly- nomial, and exponential ambiguity of finite automata with ǫ -transitions. It giv es an algorithm for testing the exponential ambiguity of an automaton A in time O ( | A | 2 E ) , and finite or polynomial ambiguity in time O ( | A | 3 E ) . These complexi- ties significantly improv e over the previou s best complexities given for the same problem. Furthermore, t he algorithms presented are simple and are based on a general algorithm for the composition or intersection of automata. W e al so gi ve an algorithm to d etermine the degree of polynomial a mbiguity of a finite automa- ton A that is polyn omially ambiguous in time O ( | A | 3 E ) . Finally , we present an application of our algorithms to an approximate computation of the entropy of a probabilistic automaton. 1 Intr oduction The question of the ambiguity o f finite automata arises in a variety of contexts. In some cases, th e app lication of an algorithm requ ires an inp ut autom aton to be finitely ambigu- ous, in other s th e conver genc e of a bou nd o r guarantee relies on that finite ambig uity or the asymptotic rate of the increase of ambiguity as a functio n of the string length. Thus, in all these cases, one needs an alg orithm to test the amb iguity , either to determin e if it is finite, or to estimate its asymptotic rate of increase. The pro blem of testing ambigu ity has been extensi vely analyzed in the p ast. The problem of d etermining the degree of ambiguity of an auto maton with finite ambigu- ity was shown to b e PSP ACE-complete. Howe ver , testing finite amb iguity can be do ne in p olynomial time using a characteriza tion o f po lynomial an d expo nential am biguity giv en b y [6 , 5, 9, 4, 11] . The most efficient algorithms fo r testing polyn omial and ex- ponen tial am biguity , an d thereby testing finite ambigu ity were presen ted by [1 0, 12]. The algo rithms presented in [ 12] assume th e input auto maton to be ǫ - free, but they are extended to th e case wh ere the auto maton h as ǫ -tran sitions in [10]. In the pr esence o f ǫ -transitions, the com plexity of th e algorithms giv en by [10 ] is O (( | A | E + | A | 2 Q ) 2 ) for testing the exponen tial am biguity of an automa ton A and O (( | A | E + | A | 2 Q ) 3 ) for testing polyno mial ambigu ity , wh ere | A | E stands for the numbe r of tran sitions a nd | A | Q the number of states of A . ⋆ This author’ s ne w address is: Google Research, 76 Ninth A venue, New Y ork, N Y 10 011. This p aper presents significan tly mo re efficient alg orithms fo r testing finite, poly - nomial, an d expon ential ambiguity fo r the general case of au tomata with ǫ -tran sitions. It gives an algorithm for testing the exponential ambigu ity of an au tomaton A in time O ( | A | 2 E ) , and finite or p olynomial ambigu ity in time O ( | A | 3 E ) . Th e main idea be hind our algorithms is to make use of the compo sition or intersection of finite automata with ǫ -transitions [8, 7]. The ǫ -filter used in these algorith ms crucially helps in the analy- sis and test of the amb iguity . W e also g i ve an algo rithm to d etermine the degree o f polyno mial ambiguity of a finite auto maton A that is p olynomially amb iguous in time O ( | A | 3 E ) . Fin ally , we present an applica tion of ou r algorithm s to an ap proximate com- putation of the entropy of a probabilistic automaton. The r emainder of th e p aper is o rganized as follows. Section 2 presents gener al au- tomata and a mbiguity d efinitions. In Section 3 we g i ve a brief description of existing characterizatio ns for the ambigu ity o f autom ata and extend them to the case o f au tomata with ǫ -tran sitions. In S ection 4 we p resent our algorithms for testing the finite, p olyno- mial, and exponential a mbiguity , and th e proo f of their cor rectness. Sectio n 5 details the relev ance of these algorithm s to the ap proximation o f the entr opy of probabilistic automata. 2 Pr eliminaries Definition 1. A finite auto maton A is a 5-tuple ( Σ , Q, E , I , F ) whe r e: Σ is a fi nite alphab et; Q is a fin ite set of states; I ⊆ Q the set of initial state s; F ⊆ Q the set of final states; and E ⊆ Q × ( Σ ∪ { ǫ } ) × Q a finite set of t ransitions, wher e ǫ denotes the empty string. W e denote by | A | Q the n umber of states, by | A | E the num ber of transitions an d by | A | = | A | E + | A | Q the size of an au tomaton A . Given a state q ∈ Q , E [ q ] deno tes the set of tran sitions leaving q . For two subsets R ⊆ Q and R ′ ⊆ Q , we denote by P ( R, x, R ′ ) the set o f all paths from a state q ∈ R to a state q ′ ∈ R ′ labeled with x ∈ Σ ∗ . W e also denote by p [ π ] the origin state, by n [ π ] th e destination state, an d b y i [ π ] ∈ Σ ∗ the label of a path π . A strin g x ∈ Σ ∗ is ac cepted b y A if it labels a succe ssful path, i.e. a pa th f rom an initial state to a final state. A fin ite automaton A is trim if every s tate of A be longs to a successful path. A is unambiguous if for a ny string x ∈ Σ ∗ there is at most one success- ful path labeled by x in A , otherwise, A is said ambig uous . The de gr ee of am biguity of a string x in A , d enoted by da ( A, x ) , is th e number of successful paths in A labeled by x . Note that if A co ntains an ǫ -cycle, there exist x ∈ Σ ∗ such that da ( A, x ) = ∞ . Using a d epth-first search restricted to ǫ -tran sitions, it can be decided in linear time whether A has ǫ -cycles. Thus, in the following, we will assume without loss of gene rality that A is ǫ -cycle free. The degr ee of amb iguity of A is defined as da ( A ) = sup x ∈ Σ ∗ da ( A, x ) . A is said finitely ambigu ous if da ( A ) < ∞ and infinitely amb iguous if d a ( A ) = ∞ . A is said polynomia lly a mbiguous if there exists a p olynomial h in N [ X ] such th at da ( A, x ) ≤ h ( | x | ) for all x ∈ Σ ∗ . The m inimal d egree of such a polyno mial is called th e degr e e of polynomia l ambiguity of A , denoted by dp a ( A ) . By defin ition, dp a ( A ) = 0 if f A is p v v p v q v v (a) (b) p 1 v 1 q 1 v 1 v 1 p 2 u 2 v 2 q 2 v 2 v 2 p d u d v d q d v d v d (c) Fig. 1. Illustration of the (a) (ED A), (b) (ID A) and (c) (ID A d ) properties. finitely amb iguous. When A is infinitely ambig uous but not po lynomially am biguous, we say that A is e xpone ntially ambiguous and that dpa ( A ) = ∞ . 3 Characterization of infinite ambiguity The char acterization and test of fin ite, poly nomial, and exponen tial ambiguity of finite automata without e -transitions are based on th e following f undamental prop erties. [6, 5, 9, 4, 11, 10, 1 2]. Definition 2. The following a r e thr ee k ey pr operties for the characterization of the am- biguity of an automa ta A . (a) (ED A): The r e exis ts a state q with at least two d istinct cycles la beled by some v ∈ Σ ∗ (F igur e 1(a)). (b) (ID A): Ther e e xist two distinct st ates p and q with paths labeled with v fr om p to p , p to q , and q to q , for some v ∈ Σ ∗ (F igur e 1(b)). (c) (ID A d ): Ther e exist 2 d states p 1 , . . . p d , q 1 , . . . , q d in A and 2 d − 1 strings v 1 , . . . , v d and u 2 , . . . u d in Σ ∗ such that fo r a ll 1 ≤ i ≤ d , p i 6 = q i and P ( p i , v i , p i ) , P ( p i , v i , q i ) and P ( q i , v i , q i ) ar e non- empty and for all 2 ≤ i ≤ d , P ( q i − 1 , u i , p i ) is non-emp ty (F igur e 1(c)). Observe that (EDA) im plies (ID A). Assuming (ED A), let e an d e ′ be the first tran sitions that differ in the two cycles at state q , th en we must have n [ e ] 6 = n [ e ′ ] since th e definition 1 disallows m ultiple transitions between the same tw o states with the same label. Thus, (ID A) holds for the pair ( n [ e ] , n [ e ′ ]) . In the ǫ -f ree case, it was shown that a trim a utomaton A satisfies (IDA) iff A is infinitely ambig uous [11, 1 2], tha t A satisfies (EDA) iff A is expo nentially amb iguous [4], and that A satisfies (ID A d ) iff dpa ( A ) ≥ d [10, 12]. These character izations can be straightfor wardly extended to the case of automata with ǫ -transitions in th e following propo sition. Proposition 1. Let A be a trim ǫ - cycle fr ee fin ite automaton. (i) A is infinitely ambiguou s iff A satisfies (IDA) . (ii) A is e xpo nentially ambig uous if f A satisfies (ED A) . (iii) dpa ( A ) ≥ d iff A satisfies (ID A d ) . Pr oof. The proof is by induction on the numb er of ǫ -transitions in A . If A do es no t have any ǫ -transitions, then the prop osition hold s as shown in [11, 12 ] for (i), [4] for (ii) and [12] for (iii). Assume now that A has n + 1 ǫ -transitions, n ≥ 0 , and that the statement of the propo sition h olds f or all automata with n ǫ -transition s. Select an ǫ -tran sition e 0 in A , and let A ′ be the fin ite au tomaton obtained after ap plication of ǫ -removal to A lim- ited to transition e 0 . A ′ is obtained by deleting e 0 from A and by adding a transition ( p [ e 0 ] , l [ e ] , n [ e ]) for every transition e ∈ E [ n [ e 0 ]] . It is clear th at A and A ′ are equiva- lent an d that there is a lab el-preserving bijection betwe en the paths in A and A ′ . Th us, (a) A satisfies (ID A) (r esp. (ED A), (ID A d )) if f A ′ satisfies (ID A) (resp. (ED A), (I D A d )) and (b) for all x ∈ Σ ∗ , da ( A, x ) = da ( A ′ , x ) . By induction , propo sition 1 holds for A ′ and thus, it follows from (a) and (b) that pro position 1 als o h olds for A . ⊓ ⊔ These characterizatio ns have b een used in [10, 12] to design algorithms for testing infi- nite, polyn omial, and exponential amb iguity , an d fo r co mputing the degree of polyn o- mial ambiguity in the ǫ -free case. Theorem 1 ( [10, 12]). Let A be a trim ǫ -fr ee finite automato n. 1. It is decidable in time O ( | A | 3 E ) whether A is infinitely ambiguou s. 2. It is decidable in time O ( | A | 2 E ) whether A is e xponen tially ambiguous. 3. The de gr ee of polynomial ambiguity of A , dpa ( A ) , can be computed in O ( | A | 3 E ) . The first result of theorem 1 has also bee n gene ralized by [10] to th e case of autom ata with ǫ -transitions but with a significantly w orse comp lexity . Theorem 2 ( [10]). Let A be a tr im ǫ -cycle fr ee finite automaton. It is decidable in time O (( | A | E + | A | 2 Q ) 3 ) whether A is infinitely ambiguous. The m ain idea u sed in [10] is to defined f rom A an ǫ -free autom aton A ′ such th at A is infinitely ambigu ous iff A ′ is infinitely ambigu ous. Howe ver , the n umber of transitions of A ′ is | A | E + | A | 2 Q . This explains why the complexity in the ǫ - transition case i s s ignif- icantly worse than in th e ǫ -free case. A similar ap proach can be u sed straightforwardly to test the exp onential amb iguity of A with complexity O (( | A | E + | A | 2 Q ) 2 ) and to co m- pute dpa ( A ) when A is p olynomially ambiguou s with co mplexity O (( | A | E + | A | 2 Q ) 3 ) . Note that we give here tighter estimates of the complexity of the algor ithms of [ 10, 12] where the authors gave complexities using the loose inequality: | A | E ≤ | Σ | · | A | 2 Q . 4 Algorithms Our algorithms for testing ambiguity are based on a gener al algorithm for the compo si- tion o r intersection o f automata, wh ich we d escribe in th e following section b oth to b e self-containe d, and to give a pro of of the co rrectness of th e ǫ -filter which we have no t presented in earlier publication s. 0 1 b b 2 b 3 b b a 0 1 b b 2 a 3 a b 0,0 1,1 b 0,1 b 2,1 b 3,1 b b b 3,2 a 3,3 a (a) (b) (c) Fig. 2. Example of finite automaton intersection. (a) Finite automata A 1 and (b) A 2 . (c) Result of the intersection of A 1 and A 2 . 4.1 Intersection of finite automata The intersection of fin ite automata is a special case of th e general comp osition algor ithm for weighted transduc ers [8, 7 ]. States in the intersection A 1 ∩ A 2 of two finite automata A 1 and A 2 are identified with pa irs o f a state of A 1 and a state of A 2 . Leaving aside ǫ -transitions, the following rule specifies how to comp ute a transition of A 1 ∩ A 2 from approp riate transitions of A 1 and A 2 : ( q 1 , a, q ′ 1 ) and ( q 2 , a, q ′ 2 ) = ⇒ (( q 1 , q ′ 1 ) , a, ( q 2 , q ′ 2 )) . (1) Figure 2 illustrates th e algor ithm. A state ( q 1 , q 2 ) is initial (re sp. final) wh en q 1 and q 2 are initial (resp. final). In the worst case, all transition s of A 1 leaving a state q 1 match all those o f A 2 leaving state q 2 , thus the sp ace and time co mplexity of composition is quadra tic: O ( | A 1 || A 2 | ) , or O ( | A 1 | E | A 2 | E ) when A 1 and A 2 are trim. Epsilon filtering A straigh tforward gen eralization of the ǫ -f ree case would gener ate redund ant ǫ -path s. T his is a crucial issue in the more general case of the in tersection of weigh ted automata over a non-id empotent semiring, since it would lead to an inc or- rect r esult. T he weigh t of two match ing ǫ -path s of the original au tomata would then be counted as m any tim es as the num ber o f redun dant ǫ -paths gen erated in th e result, instead of one. It is also a crucial problem in the unweighted case that we are consider- ing since redundan t ǫ -paths can affect the test o f infinite amb iguity , as we shall see in the next section . A critical compo nent of the composition algor ithm o f [8, 7] consists howe ver of precisely cop ing with this prob lem using a metho d called epsilon filtering . Figure 3(c) illustrates the pro blem just men tioned. T o match ǫ -paths lea ving q 1 and those leaving q 2 , a generaliza tion o f the ǫ - free intersectio n can make the following moves: (1) first move forward on an ǫ -tr ansition o f q 1 , or even a ǫ - path, an d stay at the same state q 2 in A 2 , with the h ope of later finding a tran sition whose lab el is some label a 6 = ǫ matching a transition o f q 2 with the same label; (2 ) pro ceed similarly by following an ǫ -tran sition or ǫ -path leaving q 2 while stayin g at the same state q 1 in A 1 ; or , (3) match an ǫ -transition of q 1 with an ǫ -tran sition of q 2 . Let us renam e existing ǫ -labels of A 1 as ǫ 2 , and existing ǫ -labels of A 2 ǫ 1 , and let us a ugment A 1 with a self-lo op labeled with ǫ 1 at all states and similar ly , augment A 2 with a s elf-loo p lab eled with ǫ 2 at all states, as illustrated by Figures 3(a) and (b ). Th ese ε 1 ε 2 a ε 2 ε 1 b (0 , 0) (0 , 1) (0 , 2) (1 , 0) (1 , 1) (1 , 2) (2 , 0) (2 , 1) (2 , 2) ǫ 1 : ǫ 1 ǫ 1 : ǫ 1 ǫ 1 : ǫ 1 ǫ 1 : ǫ 1 ǫ 1 : ǫ 1 ǫ 1 : ǫ 1 ǫ 2 : ǫ 2 ǫ 2 : ǫ 2 ǫ 2 : ǫ 2 ǫ 2 : ǫ 2 ǫ 2 : ǫ 2 ǫ 2 : ǫ 2 ǫ 2 : ǫ 1 ǫ 2 : ǫ 1 ǫ 2 : ǫ 1 ǫ 2 : ǫ 1 0 ε2:ε1 x:x 1 ε1:ε1 2 ε2:ε2 x:x ε1:ε1 x:x ε2:ε2 (a) (b) (c) (d) Fig. 3. Marking of automata, redundant paths and filter . (a) ˜ A 1 : self-loop labeled with ǫ 1 added at all states of A 1 , regular ǫ s r enamed to ǫ 2 . (b) ˜ A 2 : self-loop labeled with ǫ 2 added at all states of A 2 , regular ǫ s renamed to ǫ 1 . (c) Redundant ǫ -paths: a st raightforw ard generalization of the ǫ -free case could generate all the paths from (0 , 0) to ( 2 , 2) for example, ev en when composing just two simple transducers. (d) Filter transducer M allo wing a unique ǫ -path. self-loops co rrespond to staying at the sam e state in that machine while co nsuming an ǫ -label of th e other tran sition. The three moves just describ ed now co rrespond to the matches (1) ( ǫ 2 : ǫ 2 ) , (2) ( ǫ 1 : ǫ 1 ) , an d ( 3) ( ǫ 2 : ǫ 1 ) . The g rid of Figure 3(c) shows all the possible ǫ -paths between intersection states. W e will deno te by ˜ A 1 and ˜ A 2 the autom ata obtained after application of these chang es. For the r esult of in tersection not to be redu ndant, be tween any two o f these states, all but on e path must b e d isallo wed. Th ere are ma ny possible ways of selecting that p ath. One natur al way is to select the shorte st path with th e diagonal transition s ( ǫ -matchin g transitions) taken first. Figure 3(c) illustrates in bo ldface the path just describ ed fr om state (0 , 0) to state (1 , 2) . Remarkably , th is filterin g mechan ism itself can be enco ded as a finite-state transducer such a s the transducer M of Figure 3(d). W e de note by ( p, q )  ( r, s ) to indicate that ( r , s ) can be reached from ( p, q ) in the grid . Proposition 2. Let M be the transdu cer of F igur e 3(d). M allows a unique path be- tween any two states ( p, q ) an d ( r , s ) , with ( p, q )  ( r, s ) . Pr oof. Let a d enote ( ǫ 1 : ǫ 1 ) , b d enote ( ǫ 2 : ǫ 2 ) , c denote ( ǫ 2 : ǫ 1 ) , and let x stand f or any ( x : x ) , with x ∈ Σ . The follo wing sequen ces must be disallowed by a shortest-p ath filter with matching transitions first: ab, ba, ac, bc . This is because, from any state, instead of the moves ab o r ba , th e matching or diagon al tr ansition c can be taken. Similarly , instead of ac or bc , ca an d cb can be taken for an earlier match . Conv ersely , it is clear from the grid o r an immedia te recursion th at a filter d isallo wing these seq uences accepts a u nique path between two connected states of the grid. Let L be the set o f sequenc es over σ = { a, b, c, x } th at contain o ne of the disallowed sequence ju st men tioned as a sub string that is L = σ ∗ ( ab + b a + ac + bc ) σ ∗ . Then L represents exactly th e set of p aths allowed by that filter and is th us a r egular language. Let A be an autom aton representing L (Figure 4(a)). An automaton represen ting L can 0 a b c x 1 a 2 b 3 b c a c a b c x {0} c x {0,1} a {0,2} b x a {0,3} b c x b c a a b c x 0 c x 1 a 2 b x a 3 b c x b c a a b c x (a) (b) (c) Fig. 4. (a) Fi nite automaton A r epresenting the set of disallowed sequences. (b) Automaton B , result of the determinization of A . Subsets are indicated at each state. (c) Au tomaton C obtained from B by complementation, state 3 is no t coaccessible. be constru cted f rom A by determinization and complementation ( Figures 4(a)- (c)). Th e resulting automaton C is equivalent to the tran sducer M a fter removal of th e state 3 , which does not admit a path to a final state. ⊓ ⊔ Thus, to in tersect two finite au tomata A 1 and A 2 with ǫ -transitions, it suffices to com- pute ˜ A 1 ◦ M ◦ ˜ A 2 , using the the ǫ -free rules of intersection or compo sition. Theorem 3. Let A 1 and A 2 be two finite automa ta with ǫ - transitions. T o each pair ( π 1 , π 2 ) o f successful p aths in A 1 and A 2 sharing the sa me input la bel x ∈ Σ ∗ corr e- sponds a uniqu e successful path π in A 1 ∩ A 2 labeled by x . Pr oof. This f ollows straightfor wardly fr om proposition 2. ⊓ ⊔ 4.2 T es ting for infinite ambiguity W e start with a test of the exponen tial a mbiguity of A . The ke y is that the (EDA) prop- erty translates into a very simple property for A 2 = A ∩ A . Lemma 1. Let A be a trim ǫ - cycle fr ee finite automato n. A satisfies (EDA) iff there exis ts a str o ngly conn ected co mponent of A 2 = A ∩ A that contains two states o f th e form ( p, p ) an d ( q, q ′ ) , wher e p , q and q ′ ar e states of A with q 6 = q ′ . Pr oof. Assume that A satisfies ( ED A). There exist a state p an d a string v such tha t there ar e two distinct cycles c 1 and c 2 labeled by v at p . Le t e 1 and e 2 be the first edges that dif fer in c 1 and c 2 . W e can then write c 1 = πe 1 π 1 and c 2 = πe 2 π 2 . If e 1 and e 2 share th e same label, let π ′ 1 = π e 1 , π ′ 2 = π e 2 , π ′′ 1 = π 1 and π ′′ 2 = π 2 . If e 1 and e 2 do n ot share the same label, exactly o ne of the m m ust be an ǫ -transition . By symmetry , we ca n a ssume with out loss o f g enerality th at e 1 is the ǫ -tr ansition. Let π ′ 1 = π e 1 , π ′ 2 = π , π ′′ 1 = π 1 and π ′′ 2 = ǫ 2 π 2 . In both cases, let q = n [ π ′ 1 ] = p [ π ′′ 1 ] and q ′ = n [ π ′ 2 ] = p [ π ′′ 2 ] . Observe that q 6 = q ′ . Since i [ π ′ 1 ] = i [ π ′ 2 ] , π ′ 1 and π ′ 2 are matched by intersectio n resulting in a path in A 2 from ( p, p ) to ( q , q ′ ) . Similarly , since i [ π ′′ 1 ] = i [ π ′′ 2 ] , π ′′ 1 and π ′′ 2 are matched by intersection resulting in a path fro m ( q , q ′ ) to ( p, p ) . T hus, ( p, p ) and ( q , q ′ ) are in the same strongly connected componen t of A 2 . Con versely , assum e that there exist states p , q and q ′ in A such that q 6 = q ′ and that ( p, p ) and ( q , q ′ ) are in the same strongly co nnected compone nt of A 2 . Let c be a cycle in ( p, p ) going thro ugh ( q , q ′ ) , it has been obtaine d by matching two cycles c 1 and c 2 . If c 1 were equal to c 2 , intersection would match these two paths creating a path c ′ along which all the states would be of the f orm ( r , r ) , and since A is trim this would contradict Theore m 3. Thus, c 1 and c 2 are distinct and (ED A) holds. ⊓ ⊔ Lemma 1 leads to a straightfor ward a lgorithm for testing e xpo nential ambig uity . Theorem 4. Let A b e a trim ǫ -cycle fr e e finite automaton . It is dec idable in time O ( | A | 2 E ) whether A is exponentially ambig uous. Pr oof. The algorithm pro ceeds as follows. W e co mpute A 2 and, u sing a depth- first search of A 2 , trim it and co mpute its stron gly connected co mponents. I t follows from Lemma 1 that A is exponentially ambiguo us iff there is a strong ly connected comp onent that con tains two states o f the fo rm ( p, p ) and ( q , q ′ ) with q 6 = q ′ . Findin g such a strongly co nnected comp onent can be done in time linea r in the size of A 2 , i. e. in O ( | A | 2 E ) since A an d A 2 are trim. T hus, the co mplexity of the alg orithm is in O ( | A E | 2 ) . ⊓ ⊔ T esting the ( ID A) pro perty requires findin g three paths shar ing the same la bel in A . T his can be done in a natural way using the automaton A 3 = A ∩ A ∩ A , as shown belo w . Lemma 2. Let A be a trim ǫ -cycle fr ee finite automa ton. A satisfies (ID A) iff there exist two distinct s tates p and q in A with a non- ǫ path in A 3 = A ∩ A ∩ A fr om state ( p, p, q ) to state ( p, q , q ) . Pr oof. Assume that A satisfies (IDA). Then, there exists a string v ∈ Σ ∗ with th ree paths π 1 ∈ P ( p, v , p ) , π 2 ∈ P ( p, v , q ) and π 3 ∈ P ( q , v , p ) . Since these three p aths share the same lab el v , th ey are matched by intersection r esulting in a path π in A 3 labeled with v from ( p [ π 1 ] , p [ π 2 ] , p [ π 3 ]) = ( p, p , q ) to ( n [ π 1 ] , n [ π 2 ] , n [ π 3 ]) = ( p, q , q ) . Con versely , if there is a no n- ǫ p ath π form ( p, p, q ) to ( p, q , q ) in A 3 , it has bee n obtained by matching three paths π 1 , π 2 and π 3 in A with the same input v = i [ π ] 6 = ǫ . Thus, (IDA) holds. ⊓ ⊔ Finally , Theor em 4 and Lemm a 2 can be com bined to yield the follo wing result. Theorem 5. Let A b e a trim ǫ -cycle fr e e finite automaton . It is dec idable in time O ( | A | 3 E ) whether A is finitely , polynomially , or exponentially ambiguous. Pr oof. First, Th eorem 4 can be used to test whether A is exponentially amb iguous b y computin g A 2 . The comp lexity o f this step is O ( | A | 2 E ) . If A is not expo nentially ambiguo us, we proceed by computing and trim ming A 3 and then testing wh ether A 3 verifies the p roperty descr ibed in lemm a 2. This is don e by co nsidering the au tomaton B on the a lphabet Σ ′ = Σ ∪ { # } obtain ed from A 3 by adding a transition labeled by # fro m state ( p, q , q ) to state ( p, p, q ) for every pair ( p, q ) of states in A such that p 6 = q . It f ollows that A 3 verifies the condition in lem ma 2 iff there is a cycle in B containing bo th a tr ansition lab eled by # and a tra nsition labeled by a sy mbol in Σ . This pro perty can be che cked straightfor wardly using a depth-first search o f B to comp ute its strongly co nnected com ponents. If a stron gly conne cted compon ent o f B is fou nd that contains both a transition labeled with # and a transition labeled by a symbo l in Σ , A verifies (IDA) b ut n ot (EDA) and th us A is polyn omially ambiguo us. Oth erwise, A is finitely ambiguo us. The complexity of this step is linear in the size of B : O ( | B | E ) = O ( | A E | 3 + | A Q | 2 ) = O ( | A E | 3 ) since A and B are trim. The total complexity of the algorithm is O ( | A | 2 E + | A | 3 E ) = O ( | A | 3 E ) . When A is polyn omially a mbiguous, we can derive fro m th e algorithm ju st described one that compu tes dpa ( A ) . Theorem 6. Let A b e a t rim ǫ -cycle fr ee finite auto maton. If A is polyno mially ambigu- ous, dpa ( A ) can be computed in time O ( | A | 3 E ) . Pr oof. W e first compu te A 3 and use the a lgorithm of theore m 5 to test whether A is polyno mially ambigu ous and to compu te all the pairs ( p, q ) that verify the condition of Lemma 2. This step has complexity O ( | A | 3 E ) . W e then compute the comp onent g raph G of A , and for ea ch pair ( p, q ) fou nd in the previous step, w e add a tran sition labeled with # fro m the stro ngly conn ected compo- nent of p to the one of q . If there is a path in tha t gr aph co ntaining d ed ges lab eled by # , then A verifies (ID A d ). T hus, dpa ( A ) is the maximum nu mber of ed ges marked by # that can be found along a path in G . Since G is acyclic, this number ca n be computed in linear time in the size o f G , i.e. in O ( | A | 2 Q ) . Thus, the overall com plexity of the al- gorithm is O ( | A | 3 E ) . ⊓ ⊔ 5 Ap plication to the Appr oximation of Entr opy In this section , we describe an ap plication in which determ ining the degree of ambigu- ity of a pr obab ilistic au tomaton helps estimate the qua lity o f an appro ximation of its entropy . W eighted autom ata are autom ata in wh ich each tran sition carries some weight in addition to the usual alphabet symbo l. The weights are elements of a semiring, that is a ring that may lack negation. The follo wing is a more formal definition. Definition 3. A weighted au tomaton A over a semiring ( K , ⊕ , ⊗ , 0 , 1) is a 7- tuple ( Σ , Q, I , F, E , λ, ρ ) wher e: Σ is the finite alph abet of the automa ton, Q is a fin ite set of states, I ⊆ Q the set of in itial states, F ⊆ Q the set of fina l states, E ⊆ Q × Σ ∪ { ǫ } × K × Q a finite set of transitions, λ : I → K the initial weight fu nction mapping I to K , an d ρ : F → K the fin al w eight functio n mapping F to K . Giv en a transition e ∈ E , we d enote by w [ e ] its weight. W e extend the weight func tion w to p aths by defin ing th e weight o f a path as the ⊗ -produ ct of the weights of its constituent tran sitions: w [ π ] = w [ e 1 ] ⊗ · · · ⊗ w [ e k ] . The weight associated by a weighted automaton A to an input string x ∈ Σ ∗ is defined by: [ [ A ] ]( x ) = M π ∈ P ( I ,x ,F ) λ [ p [ π ]] ⊗ w [ π ] ⊗ ρ [ n [ π ]] . (2) The entropy H ( A ) of a probabilistic automaton A is defined as: H ( A ) = − X x ∈ Σ ∗ [ [ A ] ]( x ) log ([ [ A ] ]( x )) . (3) Let K den ote ( R ∪{ + ∞ , −∞} ) × ( R ∪{ + ∞ , −∞} ) . The system ( K , ⊕ , ⊗ , (0 , 0) , (1 , 0)) where ⊕ and ⊗ ar e d efined as follows d efines a co mmutative semiring called the en- tr o py semiring [2]. For any two pairs ( x 1 , y 1 ) and ( x 2 , y 2 ) in K , ( x 1 , y 1 ) ⊕ ( x 2 , y 2 ) = ( x 1 + x 2 , y 1 + y 2 ) (4) ( x 1 , y 1 ) ⊗ ( x 2 , y 2 ) = ( x 1 x 2 , x 1 y 2 + x 2 y 1 ) . (5) In [2 ], the author s show that a gen eralized shor test-distance algorith m over this semir- ing cor rectly computes the entropy of an unam biguous probabilistic automaton A . The algorithm starts by m apping th e weight of eac h transition to a pair wher e the first el- ement is th e p robability and the secon d the entropy: w [ e ] 7→ ( w [ e ] , − w [ e ] log w [ e ]) . The algo rithm then pro ceeds by com puting the generalize d shortest-distance under the entr opy semiring , which co mputes the ⊕ -sum of the weigh ts of a ll accepting paths in A . In this sectio n, we sho w that the same shortest-distance algorithm yields an approx- imation of the e ntropy of an ambig uous probabilistic automaton A , where th e approxi- mation quality is a functio n of the degree of polynomial ambiguity , dpa ( A ) . Our proofs make u se of the stand ard lo g-sum in equality [ 3], a special case of Jensen’ s ine quality , which holds for any positi ve reals a 1 , . . . , a k , and b 1 , . . . , b k : k X i =1 a i log a i b i ≥ k X i =1 a i ! log P k i =1 a i P k i =1 b i . (6) Lemma 3. Let A be a pr obab ilistic automa ton a nd let x ∈ Σ + be a string ac cepted by A o n k paths π 1 , . . . , π k . Let w ( π i ) b e the pr obab ility of path π i . Clearly , [ [ A ] ]( x ) = P k i =1 w ( π i ) . Then, k X i =1 w ( π i ) log w ( π i ) ≥ [ [ A ] ]( x )(log [ [ A ] ]( x ) − log k ) . (7) Pr oof. The result fo llo ws straightfo rwardly from the log -sum ineq uality , with a i = w ( π i ) and b i = 1 : k X i =1 w ( π i ) log w ( π i ) ≥ k X i =1 w ( π i ) ! log P k i =1 w ( π i ) k = [ [ A ] ]( x ) (log[ [ A ] ]( x ) − log k ) . (8) ⊓ ⊔ For a probabilistic automaton A , let S ( A ) be the quan tity comp uted by the gener alized shortest-distance algo rithm with the en tropy semiring. For an un ambiguous automaton A , S ( A ) = H ( A ) [2] . Theorem 7. Let A b e a pr obabilistic automa ton and let L denote the expected len gth of strings accepted by A (i.e. L = P x ∈ Σ ∗ | x | [ [ A ] ]( x ) ). Then , 1. If A is finitely ambigu ous with d e gr ee of a mbiguity k (i. e. da ( A ) = k fo r some k ∈ N ), the n H ( A ) ≤ S ( A ) ≤ H ( A ) + log k . 2. If A is po lynomially amb iguous with de gr ee of polyn omial ambig uity k (i.e. dp a ( A ) = k for some k ∈ N ), then H ( A ) ≤ S ( A ) ≤ H ( A ) + k log L . Pr oof. The lower bound , S ( A ) ≥ H ( A ) follows from the observation that for a strin g x that is accepted in A by k p aths π 1 , . . . , π k , k X i =1 w ( π i ) log ( w ( π i )) ≤ ( k X i =1 w ( π i )) log( k X i =1 w ( π i )) . (9) Since the quantity − P k i =1 w ( π i ) log ( w ( π i )) is string x ’ s contribution to S ( A ) and the quantity − ( P k i =1 w ( π i )) log ( P k i =1 w ( π i )) its co ntribution to H ( A ) , summing over all accepted strings x , we obtain H ( A ) ≤ S ( A ) . Assume th at A is finitely ambig uous with d egree of amb iguity k . Le t x ∈ Σ ∗ be a string that is accepted on l x ≤ k p aths π 1 , . . . , π l x . By Lemma 3, l x X i =1 w ( π i ) log w ( π i ) ≥ [ [ A ] ]( x )(log[ [ A ] ]( x ) − log l x ) ≥ [ [ A ] ]( x )(log[ [ A ] ]( x ) − log k ) . (10) Thus, S ( A ) = − X x ∈ Σ ∗ l x X i =1 w ( π i ) log w ( π i ) ≤ H ( A ) + X x ∈ Σ ∗ (log k )[ [ A ] ]( x ) = H ( A ) + log k . (11) This proves the first statement of the theorem . Next, assum e that A is polynomially am biguous with d egree o f polyno mial amb i- guity k . By Lemma 3, l x X i =1 w ( π i ) log w ( π i ) ≥ [ [ A ] ]( x )(log[ [ A ] ]( x ) − log l x ) ≥ [ [ A ] ]( x )(log[ [ A ] ]( x ) − log( | x | k )) . (12) Thus, S ( A ) ≤ H ( A ) + X x ∈ Σ ∗ k [ [ A ] ]( x ) log | x | = H ( A ) + k E A [log | x | ] (13) ≤ H ( A ) + k log E A [ | x | ] = H ( A ) + k log L, ( by Jensen’ s ineq uality ) which proves the second statement of the theore m. ⊓ ⊔ The quality of the approx imation o f the entropy of a probabilistic autom aton A depends on the expected length L of an accep ted string . L c an be compu ted efficiently for an arbitrary pro babilistic automaton using the expectation semiring an d the gen eralized shortest-distance algorithm s, using techniques similar to the on es described in [2]. The definition of the expectation semiring is identical to the en tropy semiring. Th e only difference is in the initial step, wh ere the weigh t of each transition in A is map ped to a pair of elements. Under the expectation semiring, the mapping is w [ e ] 7→ ( w [ e ] , w [ e ]) . 6 Conclusion W e presented simple an d efficient algorithm s for testi ng the finite, poly nomial, o r expo- nential ambigu ity of finite au tomata with ǫ -transition s. W e conjec ture that the running- time comp lexity o f our algorithms is optimal. These algorithm s have a variety of ap- plications, in particular to test a p re-conditio n for th e applicability of other auto mata algorithm s. Our ap plication to the appr oximation of the en tropy gives another illustra- tion of the applications of these algorithms. Our algorithm s also illustrate the promine nt role play ed by the g eneral algorithm for the in tersection or comp osition of automata and transdu cers with ǫ -transitions in the design of testing algo rithms . Composition can be used to d e vise simple a nd efficient testing algor ithms. W e have shown else where how it can be u sed to test the fun ctional- ity o f a finite-state tra nsducer or to test the twin s prop erty fo r weigh ted au tomata and transducer s [1]. Acknowledgments. The research of Cyril Allauzen and Mehryar Mohri wa s partially sup- ported by the Ne w Y ork State Office of Science T echno logy and Academic Research (NYS- T AR). This project was also sponsored i n part by the Department of the Army A ward Num- ber W81XWH-04-1-03 07. The U.S. Army Medical Research Acquisition Activ ity , 820 Chandler Street, Fort Detrick MD 217 02-5014 is the a warding and administering acquisition of fice. The content of this material does not necessarily reflect the position or the policy of the Governme nt and no of ficial endorsement should be inferred. Refer ences 1. Cyril Allauzen and Mehryar Mohri. Efficient Algorithms for T esting the T wins Property. J ournal of Automata, Langua ges and Combinatorics , 8(2):117–144, 2003. 2. Corinna Cortes, Mehryar Mohri, Ashish Rastogi, and Michael Ri ley . Efficient computation of the relativ e entropy of probabilistic automata. In LATIN 2006 , volume 3887 of Lecture Notes in Computer Science , pages 323–336 . Springer , 2006. 3. Thomas M . Co ver and Jo y A. T homas. Elements of Information Theory . John W iley & Sons, Inc., Ne w Y ork, 1991. 4. Oscar H. Ibarra and Bala Ravikumar . On sp arseness, ambiguity and other de cision problems for acceptors and transducers. In ST ACS 1986 , volume 210 of L ectur e Notes in Computer Science , pages 171–1 79. Springer , 1986. 5. G ´ erard Jacob . Un algorithme calculant le ca rdinal, fini ou infini, de s demi-groupes de matri- ces. Theor etical Computer Science , 5(2):183–20 2, 1977. 6. Arnaldo Mandel and Imre S imon. On finit e semigroups of matrices. Theore tical Computer Science , 5(2):101–111 , 1977. 7. Mehryar Moh ri, Fernando C. N. Pereira, and Micha el Riley . W eighted Automata in T ext and Speech Processing. In Pr oceedings of the 12th biennial Europ ean Conferen ce on Artificial Intelligence (ECAI-96) . John W iley and S ons, 1996 . 8. Fernando Pereira and Michael R iley . Finite State Languag e Pr ocessing , chapter S peech Recognition by Composition of W eighted Finite Automata. The MIT Press, 1997. 9. Christophe Reutenauer . Propri ´ et ´ es arithm ´ etiques et topologiques des s ´ eries rationnelles en v ariable non commutativ e. Th ` ese troisi ` eme cycle, Uni versit ´ e Paris VI, 1977 . 10. Andreas W eber . ¨ Uber die Mehrdeutigk eit und Wertigkeit v on endlichen, Automaten und Transducern. Dissertation, Goethe-Uni versit ¨ at F rankfurt am Main, 1987 . 11. Andreas W eber and Helmut Seidl. On the degree of ambiguity of finite automata. In MFCS 1986 , volume 23 3 of L ectur e Notes in Computer Scienc e , pages 620–62 9. Springer , 1986. 12. Andreas W eber and Helmut Seidl. On the degree of ambiguity o f finite automata. Theor etical Computer Science , 88(2):325– 349, 1991.