FORM facts

F ORM facts J.A.M. V ermasere n Nikhef, Science Park 105, 1082XG, Amsterdam, The Netherlands Some of the n ew features of the sym b olic manipulation system FORM are discussed. Then some recent results running its multithreaded v ersion TF ORM are shown. Finally the plans for th e future are presented. 1. New F eatures Over the past few y ears t he sym b olic ma nip- ulation system FORM[1] ha s pick ed up a num- ber of new fea tur es. Some are desig ne d for b et- ter sp eed, some f or more co nv enience. The most recent facility is the set of transform statements for manipulating functions with a large num b er of arguments. The pro blem her e is that do ing the arguments one b y o ne often in volves a large nu mber of sta temen ts, and always a sup erfluous amount of pattern matching. Just imag ine we hav e a function f with 20 arguments. All hav e a v alue of either zero or one. Now we wan t to re- place the zero es by ones and the ones by ze ro es. One wa y to do this is with Multip ly f1; repeat id f(x?, ?a)*f 1(?b) = f(?a)* f1(?b ,1-x) ; id f*f1( ?a) = f(?a) ; Note that the re pe a t lo op ha s to be done 20 times. With the new transform sta tement we w ould hav e Transf orm,f ,replace(1,last)=(1,0,0,1); On my laptop the first metho d takes 45 . 20 µ sec. and the seco nd metho d takes 1 . 38 µ s ec. In addi- tion the co de is easier to unders tand. This gives a smaller chance of error s. One can have different subkeys a nd one s tate- men t can hav e a whole chain of ope r ations as in CF H; L F = H(3, 4,2,6 ,1,1, 1,2); Transf orm,H ,tointe gralnotation(1,last), replac e(1,l ast)= (0,1,1,0), encode (1,la st):b ase=2; Print; .end F = H(9072 02); One can a lso split the transfo rm s tatement in v arious s tatements if one would like to s e e what happ ens: CF H; Off Statis tics; L F = H(3, 4,2,6 ,1,1, 1,2); Print "<1> %t"; Transf orm,H ,tointe gralnotation(1,last); Print "<2> %t"; Transf orm,H ,replac e(1,last)=(0,1,1,0); Print "<3> %t"; Transf orm,H ,encode (1,last):base=2; Print "<4> %t"; .end <1> + H(3,4 ,2,6, 1,1,1 ,2) <2> + H(0,0 ,1,0, 0,0,1 ,0,1,0,0,0,0,0,1,1 ,1,1,0 ,1) <3> + H(1,1 ,0,1, 1,1,0 ,1,0,1,1,1,1,1,0,0 ,0,0,1 ,0) <4> + H(907 202) The old style prog r am w ould hav e b een #:Work Space 10M Symbol x,x1,x 2; CF H,H1; L F = H(3, 4,2,6 ,1,1, 1,2); repeat id H(?a, x?!{0 ,1},? b) = H(?a,0 ,x-1, ?b); Multip ly H1; 1 2 repeat id H(x?, ?a)*H 1(?b) = H(?a)* H1(?b ,1-x); id H*H 1(?a) = H(?a); repeat id H(x1? ,x2?, ?a) = H(2*x1 +x2,? a); Print; .end F = H(9072 02); The r elev an t co de here takes 130 µ sec. while the comp osite transfor m statemen t takes 1 . 82 µ sec. In addition the last co de needs co mment ar y if one would lik e to know what it a c tually do es. The tra nsform statement a lso allows any p er- m utations of a r guments. In the exa mple here we hav e cyc lic p ermutations, the first is one back- wards and the second is t wo forw ards: CF f1,f2; S a,b,c, d,e; L F = f1(a ,b,c, d,e)* f2(3,2 *a,4, c,1,2 ,3); Transf orm,f 1,cycle(1,last)=-1; Transf orm,f 2,cycle(1,last)=+2; Print; .end F = f1(b,c ,d,e, a)*f2(2 ,3,3,2*a,4,c,1); There are also facilities for Lyndon[2] words of arguments a s this is usually messy to prog ram externally: Symbol x,x1,x 2; CF H,H1,f ; Off Statis tics; L F = H(3, 4,2,6 ,1,1, 1,2) +H(6,1 ,1,1, 2,3,4,2) +H(4,3 ,2,1, 4,3,2,1) +H(4,3 ,2,1, 4,2,2,2) +H(4,2 ,2,2, 4,3,2,1) +H(1,1 ,1,6, 2,4,3,2) +H(2,4 ,3,2, 1,1,1,6); Transf orm,H ,toLyndon>(1,last)= (f(1), f(0)) ; Print +s; .end F = + 2*H( 4,3,2 ,1,4, 2,2,2)*f(1) + H(4, 3,2,1 ,4,3, 2,1)*f(0) + 2*H( 6,1,1 ,1,2, 3,4,2)*f(1) + 2*H( 6,2,4 ,3,2, 1,1,1)*f(1) ; The term is m ultiplied b y f(1) when it is a Lyn- don w ord and b y f(0) when it is not. If we w ould hav e put just (1, 0) a t the end of the tra nsform statement the non-Ly ndo n term would hav e b e en absent in the output. The same prog r am, but no w with the or dering ‘smallest fir st’: Symbol x,x1,x 2; CF H,H1,f ; Off Statis tics; L F = H(3,4, 2,6,1, 1,1,2) +H(6,1 ,1,1, 2,3,4,2) +H(4,3 ,2,1, 4,3,2,1) +H(4,3 ,2,1, 4,2,2,2) +H(4,2 ,2,2, 4,3,2,1) +H(1,1 ,1,6, 2,4,3,2) +H(2,4 ,3,2, 1,1,1,6); Transf orm,H ,toLyndon<(1,last)= (f(1), f(0)) ; Print +s; .end F = + 2*H( 1,1,1 ,2,3, 4,2,6)*f(1) + 2*H( 1,1,1 ,6,2, 4,3,2)*f(1) + 2*H( 1,4,2 ,2,2, 4,3,2)*f(1) + H(1, 4,3,2 ,1,4, 3,2)*f(0) ; Large r uns s uffer fro m the problem that the computer may not be up that long. Je ns V ollinga has work ed at a chec kpo int facility which allows the user to make ‘snapshots’ befo r e the star t of a mo dule. If FORM cr a shes, fo r instance due to a power outage, one can restar t at the beginning of that mo dule. Currently this is still b eing debugged and tuned. F or so me a pplications it is still to o slow. There ar e also still so me childhoo d diseases, but things improve. 3 TFORM[3 ] has b een impro ved a bit. The ma s- ter nee ds far less time when there are brack ets and the brackets have b een index ed. In that case the master can tell the w orkers to deal with co mplete brack ets. F ro m that p oint on each worker is re- sp onsible for finding the terms o f the br ack ets o n its own. In the old setup, the master ha s to read all ter ms and put them in the ‘buck ets’, be fo re giving the buck ets to the workers. The sp eedup is noticeable. This still leav es the b ottlenecks at the end of the s orting. There exist a lgorithms that migh t b e able to deal with this but they are rather complicated. They are planned for a future upgrade. Last but not least: there hav e b een numerous bug fixes. F or this ma ny thanks to the p eo ple who provide me with concis e bug rep orts that allow me to catch these bugs. 2. Something to b oast ab out One of the great testjobs during the develop- men t ov er the past few years has been the expres- sion of Multiple Zeta V alue s [4,5,6] in terms of a minimal basis . This is mainly a matter of solving a system o f linear eq ua tions in which the co effi- cients in the homog eneous par t a re r ational num- ber s and the inhomog eneous par t co ntains sums and pro ducts o f ba s is elements of a lower w eight with sometimes rather bad rational co efficients. In the worst ca se the num b er of equations may run in the millions and the num b er o f unknows can b e ar ound 1 million or mo re. The worst r un thus far to ok 69 days on the 8 core s of one of the no des o f the computer in Karlsr uhe and v erified the conjecture that a new t yp e of basis element w as going to enter. One of the statistics in this pro gram: Time = 69738 .22 sec Gen erated terms = 676891 25208 14 FF Terms in output = 256391 0243 substi tutio n(8-sh)-4544 Byte s used = 615649 39480 The total num ber of generated terms in the job was 28,7 1 0,904 ,088,4 30 which is 600,0 00 terms per second p er core. The num ber o f v ariables increa ses with 2 w − 3 . Before this pro ject was star ted, the mathemati- cians had gotten to w = 1 8 and for w = 19 and w = 20 they had used matrix techniques to de- termine only the size of the basis. Now we hav e a full basis up to w = 26 a nd w = 28. F or w = 27 we still miss t wo basis elements but we can guess them. A t the s a me time we studied something discov- ered earlier by Broadhurst[7], c alled pushdowns in which bas is element s o f the MZV’s could b e ex- pressed in terms of a lternating sums with fewer indices as in: Z 6 , 4 , 1 , 1 = − 64 27 A 7 , 5 − 7967 1944 Z 9 , 3 + 1 12 ζ 4 3 + 11431 1296 ζ 7 ζ 5 − 799 72 ζ 9 ζ 3 + 3 ζ 2 Z 7 , 3 + 7 2 ζ 2 ζ 2 5 + 1 0 ζ 2 ζ 7 ζ 3 + 3 5 ζ 2 2 Z 5 , 3 − 1 5 ζ 2 2 ζ 5 ζ 3 − 18 35 ζ 3 2 ζ 2 3 − 56078 53 60810 75 ζ 6 2 with A 7 , 5 = H 7 , 5 − H − 7 , 5 . In ref [6] we mana ged to lo cate 16 of such re- lations in a combined sym b olic (FORM) and n u- merical (PSLQ) effort. In table 1 w e give the num be rs of basis elements of a certain type: the num b e r of elements belong- ing to the set of Lyndon w ords with odd integers greater than 1 (and adding up to the weight) , th e nu mber of suc h elemen ts in w hich the first t wo in- dices hav e b een low ered by one and t wo ones hav e bee n added, and finally the num ber of such e le- men ts in which the first four elements hav e b een low ered by one and 4 ones have b een added. As in H 5 , 3 , 5 , 3 , 5 , 3 , 3 H 7 , 5 , 3 , 3 , 3 , 3 , 3 → H 6 , 4 , 3 , 3 , 3 , 3 , 3 , 1 , 1 H 7 , 5 , 7 , 5 , 3 → H 6 , 4 , 6 , 4 , 3 , 1 , 1 , 1 , 1 In the left and top o f table 1 w e ha ve verified that the n umber of elements in which the first tw o in- dices hav e b een low ered by one and t wo ones hav e bee n a dded cor resp onds exactly to the num ber of pushdowns. F or the underlined num ber s we can determine a ba s is but testing explicit pushdowns is b e yond o ur reach. The num ber in the b ox co r- resp onds to the new run in which we found the 4 w/d 1 2 3 4 5 6 7 8 9 10 1 2 1 3 1 4 5 1 6 0 7 1 8 1 9 1 0 10 1 11 1 1 12 1 0 , 1 13 1 2 14 2 1 15 1 2 0 , 1 16 2 2 , 1 17 1 4 1 , 1 18 2 4 , 1 0 , 1 19 1 5 3 , 2 20 3 6 , 1 1 , 2 21 1 6 6 , 3 0 , 1 22 3 10 , 1 3 , 4 23 1 8 11 , 4 1 , 3 24 3 14 , 2 8 , 6 0 , 1 25 1 10 18 , 5 4 , 7 26 4 19 , 1 16 , 1 1 1 , 4 27 1 11 29 , 7 1 1 , 12 0,1,1 28 4 25 , 2 31 , 1 4 4 , 11 , 1 29 1 14 42 , 8 2 5 , 23 1,5,1 30 4 33 , 2 52 , 2 1 14,22,1 0,1,1 T able 1 : Num b er o f MZV basis elements as a function of weigh t and depth new basis element fo r which the first four indices are lo wered by o ne a nd four indices one ha ve b een added. The case with W = 27 , D = 9 is currently running. The fact t hat this system of basis construction and the pushdowns g ive the same num b ers is very suggestive. 3. F uture F eatures When we a re lo oking to wards the future we should first cons ider w ho are doing the work. I hav e compiled a list of p eo ple who hav e and are working at FORM during v arious s tages. It is shown in table 2. Then there are of course the b eta testers who sometimes put in muc h work to pro duce a con- cise bug rep or t or who come up with useful suggestions . A num b er of the most impo rtant ones a re E ttore Remiddi, Kostia Chetyrkin, Y ork Schr¨ oder, Thomas Hahn, T ak ahiro Ueda and Pe- ter Uwer. My ap olog ies if I forget p eople here. 5 JV 1984- now Geert Ja n v an Olden b org h manual(90’s) Andre Heck manual(90’s) Alber t Retey 1997- 2000 Denn y Fliegner 1998- 2000 Markus F ra nk 2000 Andrei Onishchenko 2000- 2002 Misha T ent yuko v 2002- now Jens V ollinga 2007- now Thomas Reiter 2008- now Irina Pushkina 2009- now Jan Kuip ers 2009- now T able 2 : People who ha ve w orked a t FORM 3.1. Op en Source Sometimes one would like to hav e quick pri- v ate additions for things that are ex tremely hard to prog ram a t the FORM le vel. Suc h things are often either of combinatoric na ture or spec ia l pat- terns. It is of course imp o ssible to forse e what some pe ople will need. Hence FORM should b e structured in such a wa y that it is p oss ible to make such additions ones e lf, ev en thoug h this won’t b e for beginner s. The first requirement for this is a go o d do cumentation o f the inner work- ings, including a num b er of examples. The s econd requirement is co de that can b e understo o d and is structured prop erly . Due to these tw o req uire- men ts FORM hasn’t b ee n released yet as op en source. W e hop e to b e r ectify this b y this sum- mer. Jens V ollinga is working hard at it. 3.2. Rational P olynomials Systems of equa tions that need to be solved are asking often for capa bilities with rational p olyno- mials. Most notoriously are the Lap orta[8,9] al- gorithms. This is something that FORM doe sn’t hav e curr ently . Hence it has rather high prior ity to build this in. And to build this in in a r a ther efficient w ay as belo nging to FORM. There exist libraries fo r the manipulatio n o f polynomials in a single v ariable, so me of them cla iming g r eat ef- ficiency , but there ar e no equiv alen t libraries for po lynomials in man y v ariables. In a dditio n there is the problem of notation. T oo mu ch tim e sp ent on co nv ersion will not be b eneficial. So me par - tial co de exists. Most univ ariate algor ithms (in particular the GCD) have b een implemented in v arious methods. This is b y now reaso nably fas t. F actoriza tion is completely missing. J an Kuiper s is w orking on this and also the multiv ar iate cases. It is impo rtant to deal with m ultiv ariate ratio- nal po lynomials efficiently when one likes to cre- ate a system for c o mputing Gr¨ obner bases. There are how ever several wa ys to deal with p olynomials and each w ay needs its own solution: • Small p olynomials: when they take a small amount of space they can be kept inside the argument of a function. There may b e bil- lions of s uch p olynomials. They should b e treated inside the reg ular workspace. Uni- v ariate p o lynomials will us ually b e in this category . • Intermediate p olyno mials: th ese could be handled by means of memory allo cations as is done with the dollar v ariables. One could hav e hundreds or even thousands of them. Typically not billions. • Large p olynomia ls: These are complete ex- pressions that could hav e billions of terms. Calculating their GCD would hav e to use the same mec hanisms by which expressions are treated. There s ho uld b e o nly very few of these. Symbol s x ,y; CFunct ion p acc; PolyRa tFun pacc; L F=pacc( x^2+x -3,(x+1)*(x+2))*y pacc(x ^2+3* x+1,( x+3)*(x+2))*y^2 ; Print +s; .sort F = +y*pac c(x^2 +x-3,x^2+3*x+2) +y^2*p acc(x ^2+3*x+1,x^2+5*x+6) ; id y = 1; Print; 6 .end F = pacc(2 *x^2+ 4*x-4,x^2+4*x+3); 3.3. Co de Si mplification W e like to have a wa y to in tro duce co de sim- plification. This would be r elev an t for all o utputs that would need further numerical ev aluation in the lang uages F ortra n and C. If it is p os sible we would like to extend this to the regular output for as far as factor ization is concer ned. Already some things can be done at the F ORM lev el, but this is usually r ather slow. One ca n for instance make a pro cedur e ‘tryfa c to r’ which would work like #do i = -100, 100 #call tryfact or(ac c,x+‘i’) #enddo B acc; Print; and the answer migh t b e like +acc(x -27)* acc(x+6)*acc(x+67)* (..... ..) This is how ever fa r fro m ideal. Ir ina Pushkina is working on improving things here a nd provid- ing internal co de for such simplification. 3.4. P arF ORM x TF ORM The ParFORM[10 ] subpro ject o f the Sonder- forsch ungsb ereich pro ject in K arlsruhe is coming to a close . This was lately worked at by Misha T ent yuko v. W e are consider ing asking for new funds to combine the tec hniques of P ar FORM and TFORM, so that w e c a n obtain e fficie nt run- ning on clusters of multicore machines. One ex- ample of suc h a computer is the Silicon Graphics computer a t K arlsruhe which has 24 no des, each with 8 c ores and its own hard disk of 4 Tbytes. This is still in the planning stage. Ac kno wledgments The work ha s b een supp orted by the rese a rch progra m of the D utch F oundation f or F und amen- tal Resear ch of Matter (F OM). Ma ny thanks go to the university of Ka r lsruhe a nd the DF G for the use o f their computer. REFERENC ES 1. J. A. M. V ermaseren, New fe atur es of F OR M , arXiv: math- ph/0010 025 . 2. R.C. Lyndon, On Burnsides pr oblem , T rans . Amer. Math. Soc. 77 (1 954) 202–21 5; On Burnsides pr oblem II , T r a ns. Amer. Math. So c. 78 (19 55) 329–3 32; C. Reutenauer, F r e e A lgebr as , (Calendron Press, Oxfor d, 1 993). 3. M. T ent yuko v and J. A. M. V ermaseren, The multithr e ade d version of F ORM , Com- put. P hys. Commun. 181 (201 0) 141 9-142 7, arXiv: hep-p h/07022 79 . 4. D. Za gier, V alues of zeta functions and their applic ations , in : First Eur of Mathemat- ics, V ol. I I, (Paris, 1992), Pro gr. Math., 120 , (Birkh¨ auser, pp. 497 –512 . 5. J. A. M. V e rmaseren, Harmo nic sums, Mel lin tr ansforms and inte gr als , Int . J. Mo d. Ph ys. A 14 (1 9 99) 20 37–2 076, [arXiv :hep- ph/9806 280 ]. 6. J. Bl ¨ umlein, D. J . Broadhurst and J. A. M. V ermaseren, The Multiple Zeta V alue Data Mine , Co mput. Ph ys. C o mmun . 181 (2010) 582- 625, arXiv:m ath-p h/0907255 7. 7. D. J. Broa dhurst, On the enu mer ation of irr e ducible k-fold Euler sums and their r oles in knot the ory and field the ory , arXiv: hep-t h/96041 28 . 8. S. La p o rta, In t. J . Mo d. Phys. A 15 (20 00) 5087 [arXiv:hep- ph/0102 033 ]. 9. S. Lap o rta, Ph ys. Lett. B 50 4 (20 01) 188 [arXiv:hep-ph/0 10203 2 ]. 10. M. T en tyuk ov, H. M. Staudenmaier , and J .A. M. V ermaseren ParF ORM: R e c ent development , Nucl. Instr um. Meth. A559 (2006) 224- 228