Rotation invariants of two dimensional curves based on iterated integrals

Rotation inva riants of t w o dimensional curves based on iterated integrals Josc ha Diehl, TU Berlin ∗ Octob er 15, 2018 W e in tro duce a no vel class of rotation in v ariants of t wo dimensional curv es based on iter ate d inte gr als . The in v ariants we presen t are in some sense complete and we describ e an algorithm to calculate them, giving explicit computations up to order six. W e presen t an application to online (stroke-tra jectory based) character recognition. This seems to b e the first time in the literature that the use of iterated in tegrals of a curv e is prop osed for (in v ariant) feature extraction in machine learning applications. 1 Intro duction Data that can b e represen ted as a curve in tw o dimensional Euclidean space app ears in many areas of pattern analysis. The b oundary (or silhouette) of an ob ject in an image or video recording can b e stored as a (closed) tw o dimensional curve. This has b een used for example for ob ject recognition of everyda y ob jects [ 31 ], the mapping of (aerial) photographs to terrain maps [ 30 , 36 ] and to solving jigsa w puzzles [ 17 ]. In video recordings, the path of p ersons in the field of vision can b e analyzed to detect anomalous b eha vior [ 23 , 43 ] or predict future mov es [ 5 ]. Hand gestures also naturally describ e tra jectories [ 34 ]. On a larger scale, the trav el path, usually of a p erson or vehicel, in its environmen t is easily reco vered from time stamp ed lo cation data, giv en for example either from GPS recordings or deriv ed from mobile connectivity logs. See the monograph [ 42 ] for an o verview of applications in this con text. In online (strok e-based) c haracter recognition, the input stream is usually the tra jectory of p en-mov ements, see for example the surv ey [ 33 ]. In some of these applications, there is no fixed co ordinate system in which to orient the data, so it b ecomes imp ortan t to select features of the data that are inv ariant to rotation of the input. In the example of online character recognition, thinking of multiuser tablets, the angle from whic h the device is used does not hav e to be fixed. This fact has to b e tak en in to accoun t when extracting features for the recognition task. The construction of rotation in v arian ts of images has a long history . Starting with the work of Hu [ 22 ] they are usually based on (centered) moments of the image. Hu drew the connection to the classical problem of algebraic inv ariants [ 7 ] (see [ 32 ] for a mo dern treatmen t) and was ∗ diehl@math.tu-b erlin.de 1 able to explicitly calculate sev en inv ariants. In subsequen t work several different metho ds for deriving these inv ariants hav e b een prop osed, among them Zernik e moments [ 39 ], the F ourier- Mellin transform [ 25 ] and Lie algebra metho ds [ 13 , 38 ]. The w ork of [ 1 , 15 ] using complex momen ts inspired the method that we present. Rotation inv ariant feature selection of tw o-dimensional curves has also b een treated in its own righ t. Among the techniques are F ourier series (of closed curv es) [ 19 , 41 , 24 ], wa velets [ 12 ], curv ature based metho ds [ 30 , 6 ] and integral in v ariants [ 29 ]. Let us also men tion the works on co dons [ 21 ] the primal curv ature sketc h [ 3 ] and on F reeman chains [ 18 ], which are usually not rotation in v ariant. The inv ariants that we present are based on the set of iterated integrals of a curve, whic h is usually denoted its signatur e . The signature as an ob ject of study was first in tro duced b y Chen [ 10 ] and he sho wed that a curv e is almost completely characterized b y it [ 11 ] (see [ 20 ] for a recent generalization). The imp ortance of iterated integrals has b y now become evident in areas suc h as con trol theory [ 14 ], ordinary differential equations and sto chastic analysis [ 27 , 28 , 16 , 4 ] The pap er is structured as follows. In Section 2 we define iterated integrals and introduce (minimal) algebraic notations to deal with them. Section 3 contain ts our main results, which giv e a means to calculate all rotation in v arian ts based on the signature. In Section 4 w e carry out explicit computations for inv ariants up to order six, taking some care of algebraic indep endence. Finally in Section 5 , as a prov e of concept, w e apply them to a simple character recognition problem. Let us briefly mention some adv antages of using iterated in tegrals for feature extraction. Cur- v ature based methods [ 30 , 6 ] rely on the computation of the second deriv ative, whereas the calculation of iterated integrals only needs the first deriv ative of the signal. The latter is, of course, a more stable pro cedure. In fact the stabilit y of iterated integrals goe s far b eyond this fact, since they can b e compute ev en for signals that are nowher e differ entiable ; think for example of the path of (a realization of ) Brownian motion. Moreov er there exists a go o d appro ximation theory that for example usually pro vides conv ergence of piecewiese linear approximations (see [ 16 ]). Although similar stabilit y prop erties are also shared by F ourier metho ds [ 19 , 41 , 24 ], it is known (Chapter 1 in [ 26 ]) that the F ourier series representation is not well suited for highly oscillatory signals. Considering a curve as an image in 2D (i.e. forgetting ab out the order in whic h it is dra wn), one can apply inv ariants used for images . Again, this will fail if the signal is highly oscillatory or o verlaps frequently . Moreo ver, it is for example almost impossible to distiguish the letter M from a rotated letter W c onsider e d as images . The corresp onding curves (as they are usually drawn) are completetely different though; the first “turn” on the letter M is to the righ t, the first one on the letter W is to the left. 2 The signature of a curve By a curve X we will from now on denote a contin uous mapping X : [0 , T ] → R 2 of b ounded v ariation. Using geometric reasoning we can immediately recognize t wo rotation inv ariants of suc h a curve. The first one is the Euclidean distance E betw een startingp oint and endpoint. The second one is the ar e a A sw ept out by the closed curve, which is obtained b y connecting starting and endpoint via a straigh t line. 2 Figure 1: Two r otation invariants ar e shown. E is the distanc e b etwe en starting and endp oint. A is the ar e a enclose d by the curve c ounte d with orientation, as denote d by the plus and minus sign. It turns out that b oth of these quantities can b e written do wn in terms of iter ate d inte gr als 1 of the curv e, E 2 = 1 2 Z T 0 Z r 0 dX 1 u dX 1 r + 1 2 Z T 0 Z r 0 dX 2 u dX 2 r A = 1 2 Z T 0 Z r 0 dX 1 u dX 2 r − 1 2 Z T 0 Z r 0 dX 2 u dX 1 r . Indeed, the first equality is just an application of the in tegration by parts form ula (Theorem 6.22 in [ 37 ]). The second equality follo ws from Green’s theorem (Theorem 10.33 in [ 37 ]). W e thus hop e to find other linear combinations of iterated integrals that also yield rotation inv ariants. Let us in tro duce some algebraic notation in order to work with the collection of these integrals. Denote by T (( R 2 )), the space of formal p ow er series in t wo non-c ommuting v ariables x 1 , x 2 . T (( R 2 )) is the (algebraic) dual of T ( R 2 ), the space of p oly onomials in x 1 , x 2 , where the pairing, denoted b y h· , ·i is defined b y declaring all monomials to b e orthonormal, e.g. h 1 + 4 . 3 x 1 + 7 . 9 x 1 x 2 − 0 . 2 x 1 x 2 x 1 + · · · , x 1 x 2 i = 7 . 9 . 1 Since X is of b ounded v ariation the integrals are well-defined using classical Riemann-Stieltjes integration (see for example Chapter 6 in [ 37 ]). This can b e pushed muc h further though. In fact the following considerations are purely algebraic and hence hold for any curve for which a sensible integration theory exists. An example is tw o dimensional Brownian motion, which is almost surely nowhere differen tiable but nonetheless admits a Stratono vich integral. 3 W e use the usual product of monomials, denoted b y · , extended to the whole space b y bilinearity . Note that · is also non-commuting. See [ 9 ] for bac kground on these spaces. W e define the signature of X to b e S ( X ) 0 ,T := X x i 1 . . . x i n Z T 0 Z r n . . . Z r 2 0 dX i 1 r 1 . . . dX i n r n , where the sum is tak en ov er all n ≥ 0 and all i 1 , . . . , i n ∈ { 1 , 2 } . F or n = 0 the summand is , for algebraic reason, tak en to b e the constant 1. Note that S ( X ) 0 ,T is an elemen t of T (( R 2 )). It was prov en in [ 11 ] (see [ 20 ] for a generalization) that the mapping X → S ( X ) 0 ,T is “almost“ one-to-one. In other words, a curve is complete ly c haracterized b y its signature (mo dulo a ”tree-lik e“ path). This fact should b e compared to the fact that the collection of all momen ts of a compactly sup- p orted densit y completely determine that densit y (mo dulo modfications on sets of zero measure), and giv es us the justification to base the analysis of a curve in R 2 en tirely on its signature. 3 Main results The aim of this w ork is to find elements in T ( R 2 ) that are inv ariant under rotations of X . T o b e sp ecific, for θ ∈ R , let ¯ X θ := R ( θ ) X :=  cos( θ ) sin( θ ) − sin( θ ) cos( θ )  X , and compute its signature S ( ¯ X θ ) 0 ,T . W e are then interested in φ ∈ T ( R 2 ) that satisfy for all θ ∈ R and all curv es X h S ( X ) 0 ,T , φ i = h S ( ¯ X θ ) 0 ,T , φ i . Definition 1. Denote such an element φ ∈ T ( R 2 ) as r otation invariant . W e are going to devise a metho d by which we will derive al l rotation in v ariants. Our approach is inspired by [ 15 ], where, in the setting of rotation inv ariant moments, it is sho wn that is useful to work in the complex plane. So from no w on we allo w elemen ts in T (( R 2 )) and T ( R 2 ) to hav e complex co efficien ts, i.e we work with T (( C 2 )) and T ( C 2 ). The pro ofs of this section can b e found in the app endix. Theorem 2. L et n ≥ 2 and i 1 , . . . , i n ∈ { 1 , 2 } b e such that # { k : i k = 1 } = # { k : i k = 2 } . (1) Then φ := c i 1 ...i n is r otation invariant, wher e c i 1 ...i n := z i 1 · z i 2 · . . . · z i n z 1 := x 1 + ix 2 z 2 := x 1 − ix 2 . 4 The just describ ed metho d suffices to find al l rotation inv ariants: Theorem 3. L et φ ∈ T ( R 2 ) b e r otation invariant. Then we c an write φ as the finite sum of the invariants given in The or em 2 . 4 Explicit construction up to o rder six Before giving explicit expressions for in v ariants up to order six we ha ve to deal with one partic- ularit y of the signature. Namely , the fact that ev ery p olynomial in elements of the signature is actually a linear function of (different) elemen ts of the signature. 4.1 Shuffle identity The shuffle pro duct  on T ( C 2 ) is commutativ e and extended by bilinearit y from its definition on monomials. The shuffle pro duct of tw o monomials consists in the sum of all p ossible wa ys of in terleaving the tw o monomials while keeping their resp ectiv e order. F or example x 1  x 2 = x 1 x 2 + x 2 x 1 x 1 x 2  x 1 x 2 = x 1 x 2 x 1 x 2 + x 1 x 1 x 2 x 2 + x 1 x 1 x 2 x 2 + x 1 x 1 x 2 x 2 + x 1 x 1 x 2 x 2 + x 1 x 2 x 1 x 2 = 2 x 1 x 2 x 1 x 2 + 4 x 1 x 1 x 2 x 2 . See [ 9 ] for a completely rigorous definition. The significance of this pro duct stems from the follo wing shuffle identity , which is prov en for example in [ 35 ]. Lemma 4. L et X : [0 , T ] → R 2 b e a p ath of b ounde d variation. Then for every a, b ∈ T ( C 2 ) we have h S ( X ) 0 ,T , a ih S ( X ) 0 ,T , b i = h S ( X ) 0 ,T , a  b i Remark 5. The shuffle identity r epr esents, on an algebr aic level, the fact that inte gr als ob ey the inte gr ation by p arts rule. F or example, taking a = x 1 , b = x 2 , we get  Z T 0 dX 1 r  ·  Z T 0 dX 2 r  = Z T 0 Z r 0 dX 1 u dX 2 r + Z T 0 Z r 0 dX 2 u dX 1 r . 4.2 Computation to order six F rom Theorem 2 it is clear that rotation in v ariants only exist on levels of even order. On the lev el of order tw o, the theorem gives the (complex) inv ariants c 12 = x 1 x 1 − ix 1 x 2 + ix 2 x 1 + x 2 x 2 c 21 = x 1 x 1 + ix 1 x 2 − ix 2 x 1 + x 2 x 2 . W e are in terested in real inv ariants, so we take real and imaginary part Re c 12 = x 1 x 1 + x 2 x 2 Im c 12 = − x 1 x 2 + x 2 x 1 Re c 21 = x 1 x 1 + x 2 x 2 Im c 21 = x 1 x 2 − x 2 x 1 . 5 A linear basis for them is I 1 := x 1 x 1 + x 2 x 2 I 2 := x 1 x 2 − x 2 x 1 , whic h we recognize (mo dulo a prefactor of 1 / 2) as the geometric inv ariants we saw at the b egin- ning of Section 2 . It is unclear whether there exist geometrical interpretations for higher order in v ariants. Pro ceeding to order four, b y the shuffle identit y (Lemma 4 below), the information given b y I 1  I 1 , I 2  I 2 and I 1  I 2 is already kno wn from the inv ariants of order tw o. So, we start by taking a basis for span R { I 1  I 1 , I 2  I 2 , I 1  I 2 } . W e then extend it to a basis for span R { I 1  I 1 , I 2  I 2 , I 1  I 2 , Re φ i , Im φ i } , where the φ i range ov er the inv ariants of order four given b y Theorem 2 (i.e. c 1122 , c 1212 , c 1221 , c 2111 , c 2121 and c 2211 ). The additional basis vectors are given b y I 3 := 1122 − 1212 − 1221 − 2121 I 4 := − 1212 + 2112 + 2211 I 5 := 1212 − 2112 − 2211 . where, to shorten notation, w e denote i 1 . . . i n := x i 1 · . . . · x i n . Rep eating the same pro cedure for order six we get I 6 := 111222 − 112122 − 112212 − 112221 − 121212 − 121221 − 122121 − 211122 − 211212 − 211221 − 212121 − 221112 − 221121 − 222111 I 7 := − 112122 − 112221 − 121122 − 121221 − 122112 − 122211 − 211122 − 211221 − 212112 − 212211 − 221112 − 221211 − 222111 I 8 := 111222 − 112122 − 112212 − 112221 − 121212 − 121221 − 122121 − 211122 − 211212 − 211221 − 212121 − 221112 − 221121 − 222111 I 9 := − 112122 + 112221 − 121122 + 121221 − 122112 + 122211 − 211122 + 211221 − 212112 + 212211 − 221112 + 221211 − 222111 I 10 := 111222 + 112122 + 112212 + 112221 + 121212 + 121221 + 122121 + 211122 + 211212 + 211221 + 212121 + 221112 + 221121 + 222111 I 11 := − 112122 − 112221 + 121122 − 121221 + 122112 − 122211 + 211122 − 211221 + 212112 − 212211 + 221112 − 221211 + 222111 I 12 := 111222 + 112122 + 112212 + 112221 + 121212 + 121221 + 122121 + 211122 + 211212 + 211221 + 212121 + 221112 + 221121 + 222111 I 13 := 112122 − 112221 − 121122 − 121221 − 122112 − 122211 − 211122 − 211221 − 212112 − 212211 − 221112 − 221211 − 222111 I 14 := 111222 + 112122 + 112212 + 112221 + 121212 + 121221 + 122121 + 211122 + 211212 + 211221 + 212121 + 221112 + 221121 + 222111 I 15 := 112122 − 112221 − 121122 − 121221 − 122112 − 122211 − 211122 − 211221 − 212112 − 212211 − 221112 − 221211 − 222111 . 6 T able 1: Pendigits classific ation err or Order 2 (2) Order 4 (5) Order 4 (8) Order 6 (15) Order 6 (28) linear 38.34 15.81 15,58 9.98 7.83 rbf 37.60 9.27 8.18 4.91 4.60 Classification error using features of different order. The num b ers in brack ets denote the num b er of features. F or order 4 and 6 tw o different sets of features were used. The low er num b er was obtained by making the features linearily and algebraically indep enden t as describ ed in Section 4 . The higher num b er was obtained using all the inv ariants given by Theorem 2 . 5 Application to cha racter recognition W e present the application to a simple classification problem on the dataset p endigits [ 2 ], which consists of handwritten digits by 44 writers, 30 of which are used for training (7493 samples) and 14 of which are used for testing (3497 samples). Input was recorded on a tablet device and hence str oke data giv en. Since some inputs consist of m ultiple strokes (the p en is lifted from the device and then touches again at a different lo cation), we hav e to conv ert the signal into a con tinuous curv e first. W e achiev e this by just connecting endp oint of the previous and startingp oin t of the next stroke by a straight line. T o v erify that our metho d is indeed rotation in v ariant, we rotated eac h digit in the test-set by a random angle. W e used supp ort vector machines with linear and rbf kernel, where for the latter w e optimized the parameters on the training set via 3-fold crossv alidation. The results are presented in T able 1 . 6 App endix The proofs of Theorem 2 and Theorem 3 follow. W e shall need, for n ≥ 0, the pro jection operator π n whic h sets all co efficients of a p olynomial or formal series to zero, except the ones b elonging to monomials of order n , which it lea ves unchanged; for example π 2 (1 + x 1 x 2 + x 2 x 1 + x 3 1 x 2 ) = x 1 x 2 + x 2 x 1 . Pr o of of The or em 2 . Let Z = Z ( X ) b e defined as Z 1 r := X 1 r + iX 2 r Z 2 r := X 1 r − iX 2 r . Let ¯ Z θ = Z ( ¯ X θ ). Then ¯ Z θ, 1 = e − iθ Z 1 , ¯ Z θ, 2 = e iθ Z 2 , and hence for i 1 , . . . , i n ∈ { 1 , 2 } we hav e h S ( ¯ Z θ ) 0 ,T , x i 1 · . . . · x i n i = Z T 0 Z r n 0 . . . Z r 2 0 d ¯ Z θ,i 1 r 1 . . . d ¯ Z θ,i n r n = Z T 0 Z r n 0 . . . Z r 2 0 e ( − 1) i 1 iθ d Z i 1 r 1 . . . e ( − 1) i n iθ d Z i n r n = e iθ (# { k : i k =2 }− # { k : i k =1 } ) h S ( Z ) 0 ,T , x i 1 · . . . · x i n i . 7 No w, on the other hand, h S ( Z )) 0 ,T , x i 1 · . . . x i n i = h S ( X )) 0 ,T , c i 1 ...i n i h S ( ¯ Z θ )) 0 ,T , x i 1 · . . . x i n i = h S ( ¯ X )) 0 ,T , c i 1 ...i n i , whic h shows that c i 1 ...i n is rotation in v ariant if it satisfies ( 1 ). Pr o of of The or em 3 . Let n ≥ 1 and let φ n := π n φ b e the pro jection on the n -th level. Since the in v ariants given in Theorem 2 are homogeneous, it is enought to show the statement for φ n . By Lemma 6 w e hav e that φ n is also rotation in v ariant and by Lemma 7 we can write φ n as φ n = X i 1 ,...,i n ∈{ 1 , 2 } a i 1 ,...,i n c i 1 ,...,i n , for some uniquely determined a i 1 ,...,i n ∈ C . Then h S ( ¯ X θ ) 0 ,T , φ n i = X i 1 ,...,i n ∈{ 1 , 2 } a i 1 ,...,i n h S ( ¯ X θ ) 0 ,T , c i 1 ,...,i n i = X i 1 ,...,i n ∈{ 1 , 2 } a i 1 ,...,i n e iθ (# { k : i k =2 }− # { k : i k =1 } ) h S ( X ) 0 ,T , c i 1 ,...,i n i . On the other hand, since φ n is rotation in v ariant, we hav e h S ( ¯ X θ ) 0 ,T , φ n i = h S ( X ) 0 ,T , φ n i = X i 1 ,...,i n ∈{ 1 , 2 } a i 1 ,...,i n h S ( X ) 0 ,T , c i 1 ,...,i n i . Com bining, we arrive at X i 1 ,...,i n ∈{ 1 , 2 }  a i 1 ,...,i n − e iθ (# { k : i k =2 }− # { k : i k =1 } ) a i 1 ,...,i n  h S ( X ) 0 ,T , c i 1 ,...,i n i = 0 , (2) for all curv es X of b ounded v ariation. W e need to sho w: a i 1 ,...,i n = 0 if # { k : i k = 2 } − # { k : i k = 1 } 6 = 0. By ( 2 ) this follows if we can sho w the existence of paths X (1) , . . . , X (2 n ) suc h that the vectors, defined as v k := ( h S ( X ( k ) 0 ,T , c 11 ... 112 i , h S ( X ( k ) 0 ,T , c 11 ... 121 i , . . . , h S ( X ( k ) 0 ,T , c 22 ... 222 i ) , k = 1 , . . . , 2 n , are linearly indep enden t. But this follows from Lemma 8 and Lemma 7 . Lemma 6. L et φ ∈ T ( R 2 ) b e r otation invariant. Then φ n := π n ( φ ) is r otation invariant for al l n ≥ 1 . Pr o of. Let N b e the order of φ . Let X b e some curve and for let the dilation b y α ∈ R b e giv en 8 as X α := αX . Then N X n =1 α i h S ( X ) 0 ,T , π n φ i = N X n =1 h S ( X α ) 0 ,T , π n φ i = h S ( X α ) 0 ,T , φ i = h S ( ¯ X θ α ) 0 ,T , φ i = N X n =1 h S ( ¯ X θ α ) 0 ,T , π n φ i = N X n =1 α i h S ( ¯ X θ ) 0 ,T , π n φ i . Since this holds for all α ∈ R we hav e for all n ≥ 1 and all curves X h S ( X ) 0 ,T , π n φ i = h S ( ¯ X θ ) 0 ,T , π n φ i . Hence π n φ is rotation in v ariant for all n ≥ 1. Lemma 7. F or every n ≥ 1 { c i 1 ,...,i n : i 1 , . . . , i n ∈ { 1 , 2 }} is a b asis for π n T ( R 2 ) . Pr o of. Denote by v i 1 ,...,i n the vector in C n corresp onding to the co efficien ts of the monomials of c i 1 ,...,i n , i 1 , . . . , i n ∈ { 1 , 2 } , where we order the monomials lexicographically . Let M n b e the n × n matrix constructed from the v i 1 ,...,i n where w e order the rows according to the lexicographically order of i 1 . . . i n . W e hav e to s ho w that M n is of full rank. This is ob viously true for n = 1. Let b e true for an arbitrary n . Then M n +1 =  M n iM n M n − iM n  , from whic h we see that M n +1 has full rank. Lemma 8. L et n ≥ 1 . Then span C { π n ( S ( X ) 0 , 1 ) : X c ontinuous and of b ounde d variation } = π n T (( C 2 )) . (3) Pr o of. It is clear by definition that the left hand side of ( 3 ) is included in π n T (( C 2 )). W e show the other direction and use ideas of Prop osition 4 in [ 8 ]. Let x i n · . . . · x i 1 ∈ π n T (( C 2 )) b e giv en. Let X b e the piecewise linear path, that results from the concatenation of the v ectors t 1 e i 1 , t 2 e i 2 up to t n e i n , where e i , i = 1 , 2 is the standard basis of R 2 . Its signature is giv en by (see for example Chapter 6 in [ 16 ]) S ( X ) 0 , 1 = exp( t n x i n ) · . . . · exp( t 1 v i 1 ) =: φ ( t 1 , . . . , t n ) , 9 where the exp onen tial function is defined by its p o wer series. Then d dt n . . . d dt 1 φ (0 , . . . , 0) = x i n · . . . · x i 1 . Com bining this with the fact that left hand side of ( 3 ) is a closed set we get that x i n · . . . · v i 1 ∈ span C { π n ( S ( X ) 0 , 1 ) : X contin uous and of b ounded v ariation } . These elemen ts span π n T (( C 2 )), whic h finishes the pro of. References [1] Abu-Mostafa, Y aser S., and Demetri Psaltis. ”Recognitive asp ects of moment inv ariants.” P attern Analysis and Machine Intelligence, IEEE T ransactions on 6 (1984): 698-706. 1 [2] Alimoglu, F evzi, and Ethem Alpaydin. ”Metho ds of com bining multiple classifiers based on differen t representations for p en-based handwritten digit recognition.” Pro ceedings of the Fifth T urkish Artificial In telligence and Artificial Neural Net w orks Symp osium (T AINN 96. 1996. 5 [3] Asada, Haruo, and Michael Brady . ”The curv ature primal sketc h.” Pattern Analysis and Mac hine Intelligence, IEEE T ransactions on 1 (1986): 2-14. 1 [4] F abrice, Baudoin. An introduction to the geometry of sto chastic flows. Imp erial College Press, 2004. 1 [5] Bennewitz, Maren, et al. ”Learning motion patterns of p eople for compliant rob ot motion.” The In ternational Journal of Rob otics Research 24.1 (2005): 31-48. 1 [6] Calabi, Eugenio, et al. ”Differential and numerically inv ariant signature curv es applied to ob ject recognition.” International Journal of Computer Vision 26.2 (1998): 107-135. 1 [7] Ca yley , Arthur. ”An introductory memoir up on quan tics.” Philosophical T ransactions of the Ro yal So ciet y of London 144 (1854): 245-258. 1 [8] Cass, Thomas, and Peter F riz. ”Densities for rough differential equations under Hrmanders condition.” Annals of Mathematics 171 (2010): 2115-2141. 6 [9] Reutenauer, Christophe. ”F ree Lie Algebras, London MathematicalSo ciety Monographs, new series, no. 7.” (1993): 153-0041. 2 , 4.1 [10] Chen, Kuo-Tsai. ”Iterated in tegrals and exponential homomorphisms.” Pro c. London Math. So c.(3). V ol. 4. 1954. 1 [11] Chen, Kuo-Tsai. ”In tegration of pathsa faithful represen tation of paths b y non-comm utative formal p o w er series.” T rans. Amer. Math. So c 89.2 (1958): 395-407. 1 , 2 [12] Ch uang, GC-H., and C-CJ Kuo. ”W a velet descriptor of planar curves: Theory and applica- tions.” Image Pro cessing, IEEE T ransactions on 5.1 (1996): 56-70. 1 [13] Dimai, Alexander. ”Rotation inv ariant texture description using general momen t inv ariants and gab or filters.” In Pro ceedings of the 11th Scandinavian Conference on Image Analysis. 1999. 1 10 [14] Fliess, Michel. ”F onctionnelles causales non linaires et indtermines non comm utatives.” Bul- letin de la so cit mathmatique de F rance 109 (1981): 3-40. 1 [15] Flusser, Jan. ”On the indep endence of rotation momen t inv ariants.” Pattern Recognition 33.9 (2000): 1405-1410. 1 , 3 [16] F riz, P eter K., and Nicolas B. Victoir. Multidimensional sto c hastic processes as rough paths: theory and applications. V ol. 120. Cambridge Univ ersity Press, 2010. 1 , 6 [17] F reeman, Herb ert, and L. Garder. ”Apictorial jigsa w puzzles: The computer solution of a problem in pattern recognition.” Electronic Computers, IEEE T ransactions on 2 (1964): 118-127. 1 [18] F reeman, Herbert. ”Computer pro cessing of line-dra wing images.” A CM Computing Surveys (CSUR) 6.1 (1974): 57-97. 1 [19] Granlund, Gsta H. ”F ourier prepro cessing for hand print character recognition.” Computers, IEEE T ransactions on 100.2 (1972): 195-201. 1 [20] Ham bly , Ben and Lyons, T erry . Uniqueness for the signature of a path of b ounded v ariation and the reduced path group. Annals of Math. 1 , 2 [21] Hoffman, D. D., and W. A. Ric hards. ”Representing smo oth plane curves for recognition: Implications for figure-ground rev ersal.” Pro ceedings of AAAI. V ol. 82. 1982. 1 [22] Hu, Ming-Kuei. ”Visual pattern recognition by moment inv ariants.” Information Theory , IRE T ransactions on 8.2 (1962): 179-187. 1 [23] Hu, W eiming, et al. ”A surv ey on visual surv eillance of ob ject motion and b eha viors.” Systems, Man, and Cyb ernetics, Part C: Applications and Reviews, IEEE T ransactions on 34.3 (2004): 334-352. 1 [24] Kuhl, F rank P ., and Charles R. Giardina. ”Elliptic F ourier features of a closed contour.” Computer graphics and image pro cessing 18.3 (1982): 236-258. 1 [25] Li, Y a jun. ”Reforming the theory of in v ariant moments for pattern recognition.” Pattern Recognition 25.7 (1992): 723-730. 1 [26] Ly ons, T. T erry J., and Zhong Qian. System control and rough paths. Oxford Universit y Press on Demand, 2002. 1 [27] Ly ons, T erry J. ”Differen tial equations driv en by rough signals.” Revista Matemtica Ib eroamericana 14.2 (1998): 215-310. 1 [28] Ly ons, T. T erry J., Mic hael. Caruana, and Thierry Lvy . Differential equations driv en b y rough paths. V ol. 1908. Berlin: Springer, 2007. 1 [29] Mana y , Siddharth, et al. ”Integral inv ariants for shape matching.” P attern Analysis and Mac hine Intelligence, IEEE T ransactions on 28.10 (2006): 1602-1618. 1 [30] Mokh tarian, F arzin, and Alan Mac kworth. ”Scale-based description and recognition of pla- nar curv es and tw o-dimensional s hapes.” Pattern Analysis and Machine Intelligence, IEEE T ransactions on 1 (1986): 34-43. 1 [31] Mokh tarian, F arzin. ”Silhouette-based isolated ob ject recognition through curv ature scale space.” P attern Analysis and Machine Intelligence, IEEE T ransactions on 17.5 (1995): 539- 544. 1 11 [32] Olv er, Peter J. Classical inv ariant theory . V ol. 44. Cambridge Universit y Press, 1999. 1 [33] Plamondon, Rjean, and Sargur N. Srihari. ”Online and off-line handwriting recognition: a comprehensiv e survey .” Pattern Analysis and Machine Intelligence, IEEE T ransactions on 22.1 (2000): 63-84. 1 [34] Psarrou, Alexandra, Shaogang Gong, and Michael W alter. ”Recognition of h uman gestures and b eha viour based on motion tra jectories.” Image and Vision Computing 20.5 (2002): 349-358. 1 [35] Ree, Rimhak. ”Lie elements and an algebra asso ciated with shuffles.” The Annals of Math- ematics 68.2 (1958): 210-220. 4.1 [36] Ro driguez, Jeffrey J., and J. K. Aggarwal. ”Matching aerial images to 3-D terrain maps.” P attern Analysis and Machine In telligence, IEEE T ransactions on 12.12 (1990): 1138-1149. 1 [37] W.Rudin. Principles of Mathematical Analysis. McGra w-Hill, New Y ork, third edition, 1976. 2 , 1 [38] Sak ata, T oshio, et al. ”A new series of rotation inv ariant moments b y Lie transformation group theory .” F rontiers in Handwriting Recognition, 2004. IWFHR-9 2004. Nin th Interna- tional W orkshop on. IEEE, 2004. 1 [39] W allin, k e, and O. Kubler. ”Complete sets of complex Zernike mome n t inv ariants and the role of the pseudoin v ariants.” P attern Analysis and Mac hine In telligence, IEEE T ransactions on 17.11 (1995): 1106-1110. 1 [40] W olfson, Haim J. ”On curve matching.” P attern Analysis and Mac hine Intelligence, IEEE T ransactions on 12.5 (1990): 483-489. [41] Zahn, Charles T., and Ralph Z. Roskies. ”F ourier descriptors for plane closed curves.” Computers, IEEE T ransactions on 100.3 (1972): 269-281. 1 [42] Zheng, Y u. Computing with spatial tra jectories. Springer Science+ Business Media, 2011. 1 [43] Zhou, Y ue, Sh uicheng Y an, and Thomas S. Huang. ”Pair-activit y classification b y bi- tra jectories analysis.” Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. IEEE, 2008. 1 12