Formal Aspects of Grid Brokering

L. Brim and J. van d e Pol (Eds.): 8th International W orkshop on Parallel and Distributed Methods in verifiCation 2009 (PDMC’09) EPTCS 14, 2009, pp. 18–31, doi:10.4204 /EPTCS .14.2 c  A. Kert ´ esz & Zs. N ´ emeth This work is licensed under the Creativ e Commons Attribution License. F or mal Aspects of Grid Br okering ∗ Attila Ker t ´ esz MT A SZT AKI 1518 Budapest, P .O. Box 63, Hungary attila.ker tesz@sztak i.hu Zsolt N ´ emeth MT A SZT AKI 1518 Budapest, P .O. Box 63, Hungary zsnemeth@s ztaki.hu Coordinatio n in distributed en vironm ents, like Grids, in volves s electing the most ap prop riate ser- vices, resources or com positions to carr y out the planned ac tivities. Such functionalities ap pear at various levels of the infrastructure and in various means forming a blurry domain, where it is hard to see ho w the participating components are related and what their rele vant pro perties are. In this paper we focus on a subset of these p roblems: resource brokering in Grid middleware. This paper aims at establishing a sem antical mo del for bro kering and related a ctivities by d efining bro kering age nts at three levels of the Grid middleware for resource, host and broker selection. Th e main con tribu- tion of this paper is the de finition and d ecompo sition of different brokering compon ents in Grids by providing a formal model using Abstract State Machines. 1 Introd uction Grid Comput ing has e mer ged from th e area of Parallel and Distribu ted Computing more th an a decade ago [14]. Since then numero us Grid projects ha ve succe eded and Grids started to flourish. Recently se ve ral service Grid middle ware solutions are used around the world [6, 7 , 18], and a wide varie ty of resour ce manageme nt tools – which are important components of grid computi ng infra structu re – are a v ailabl e and under de v elopment [15]. Resource managemen t in Grids has many asp ects and in v olv es dif feren t app roach es and means. Among those we hav e in ve stigate d brok ering and estab lished a Grid resour ce br oke ring tax onomy [12] to determine what proper ties brokers posses and what functionali ties are desired for certain tasks. This surve y shows that the curren tly av ailable Grid resource m anagemen t tools are buil t on differ ent middle ware components supporti ng differ ent proper ties and named with a b unch of acron yms – eve n the ones ha ving similar purposes. This pletho ra of approaches formed the domain of Grid resourc e management in to a grey bo x with bl urry bou ndarie s where neithe r the user s nor the researchers c an clearly see ho w these tools are r elated and w hat their relev ant prope rties are . Until the definitio ns and interr elation s are clarified, fur ther de ve lopment and i nterop erabili ty cannot be facilitat ed. Therefore , in an earlier work we aimed at an informal definition as Grid resource management anatomy in [13]. Present work can be considere d as a continuatio n, in vestigat ing ho w the area of Grid resource managemen t can be formalized and what essent ial layers, functi onaliti es can be separate d based on the formal model. A former w ork this p aper is b uilt on prese nts a formal defini tion for Grid Comput ing [16]. That time there had been se ver al definitions for Grid Computing without the ability of making a clear distincti on between Grids and other distrib uted systems. The paper concluded that Grids cannot be defined purely by their propert ies rather , their runt ime semantic s make the real dif feren ce. Based on the analysis, a formal definition was gi ven for Grid Computing rev ealing its essenti al and characterist ic functionali ties. ∗ The res earch leadin g to th ese results has received funding from the European Community’ s Sev enth Framew ork Programme FP7/2007-2013 under grant agreement 215483 (S-Cube). A. Ke rt ´ esz & Z s. N ´ emeth 19 The aim and method ology of this paper is similar: establ ishing a formal, semantic model for Grid re- source management using Abstrac t State M ach ine . W e exten d the formal m odel for Grids defined in [16] by class ifying brok ering compon ents into three cate gori es and defini ng three agents for resource managemen t at dif ferent lev els of Grid systems. W e are not aware of any other works that in vestiga te formal models speci fically for grid resour ce manager components. B ratosin et al. propos ed a reference model for Grid archit ectures based on col ored Petri nets in [4 ]. T hough they provide a definition for job schedulin g, they do not detail brokerin g steps and mechanisms at diffe rent lev els. Altenhof en et al. in vestiga ted Service Oriented Architecture s in [1], more specifically service discov ery , mediation and composition . These components hav e some similar functi onaliti es bu t this work is more focu sed on a unified, higher lev el service frame wor k, and do not exp lore reso urce manage r componen ts. B ¨ org er et al. proposed an A SM model for workflo ws in [3]. The wo rk presents workflo w interpretati ons and transitions, which are relate d to our model, b ut they stay at the applicati on lev el and do not deal with broke ring at job lev el whereas our model target s the middle ware below the app licatio n lev el. In the followin g section we gi ve a brief introduct ion of the formal Abstract State Machine method, and in Section 3 we summarize the formal m odel for Grid Computing introdu ced in [16] and describ e our modified mo del. In Sectio n 4 we p resent and d escrib e the exte nsion s for Grid brok ering compone nts and in S ection 5 w e refine agents responsible for broker and host selection. Finally Section 6 concludes the paper . 2 Abstract State Machines ASM represe nts a mathematically well founded framew ork for system design and analys is [2]. It is able not just to model a wo rking mechanism p recise ly b ut also to re vea l t he highly abs tract nature of a system, i.e. to grasp the semantics [8]. Furthermore – unlike many other state based modeling m ethods -, it can easily be tailore d to the required le vel of abstractio n. Logicians struct ures applie d in ASMs off er an exp ressi ve, flexib le and complete way of state description. The basic sets and the functions interprete d on them can be freely chosen to the require d leve l of comple xity and precision [2]. In AS M, a signature (or vo cab ulary) is a fi nite set of functio n names, each of fixed arity . Further - more, it also contains the symbols t r ue , f al se , u nd e f , = and the usual Boolean operator s. A state A of signat ure ϒ is a nonempty set X together with interpretat ions of fu nction names in ϒ on X . X is cal led t he superu ni ve rse of A . An r -ary functio n name is interprete d as a function from X r to X , a basic function of A . A 0-ary function name is interpre ted as an element of X [9]. A location of A (can be seen like the addres s of a memory cell) is a pair l = ( f , a ), where f is a function name of arity r in v ocab ulary ϒ and a is an r -tuple of element s of X . The element f ( a ) is the c onten t of locati on l . An upda te is a p air a = ( l , b ), w here l is a locatio n and b is an element of X . F iring a at state A means putting b into the location l while othe r locatio ns remain inta ct. T he result ing state is th e seq uel of A . It means th at the interpr etation of a function f at argumen t a has been modified resulting in a ne w state. ASMs are defined as a set of rules. An update rule f ( a ) : = b causes an update (( f , a ), b ), i.e. hence the interpre tation of function f on ar gument a will result b . It must be emphasized that both a and b are ev alua ted in A . The nul lary Sel f function allo ws an agent to identi fy itself among othe r agents . It is interp reted dif feren tly by dif ferent agents (that is why it is not a member of the voca b ulary .) An agent a interprets Sel f as a while an o ther agent cannot i nterpre t it as a . The S el f funct ion c annot be the subjec t of updates. A conditio nal rule R is of form 20 Formal Aspe cts of Grid Brokeri ng if c then R 1 else R 2 endif T o fire R the guard c must be examined first and whene ver it is true R 1 otherwis e, R 2 must be fired. A block of rules is a rule an d can be fired simultane ously if they are mutu ally c onsist ent. Some ap plicati ons may require addition al sp ace during their run therefor e, the res erve of a state is the (infinite) source where ne w elements can be imported from by the follo wing construct extend U by v 1 , . . . , v n with R endextend meaning that new elements are imported from the reserve and they are assigned to uni v erse U and then rule R is fired [9]. The basic sequentia l ASM model can be exte nded in vario us ways like non-d eterminis tic sequen- tial models with the choice construc t, first-or der guard express ions, one-agent parallel and m ulti-ag ent distrib uted models. A distrib uted ASM [2] consists of a finite set of single-agen t programs Π n called modules , a signature ϒ , which include s each Fun( Π n )- { Sel f } , i.e. it contains all the function names of each module b ut not the nullary Sel f funct ion, and a collection of initial states. As it can be seen, ASM states ar e represented as (modified) log icians struc tures, i.e. basic sets (uni verses) w ith functio ns in terpre ted o n them. S tructure s are modified in ASM to enable state transitio ns for modeling dynamic systems. Applyin g a step of ASM M to state (structure) A w ill produce another state A ′ on the same set of function names. If the function names and arities are fi xed , the only way of transfo rming a structur e is to chang e the v alue of some functions for some argumen ts. Therefore, the most general struc ture tran sformatio n (AS M rule) is a guarded dest ructi ve assign ment to functions at gi ve n arg uments [2]. Refinement [ 2] is defined as a proc edure where abstract and more concrete ASMs are relat ed accor d- ing to the hierarchic al system design . At higher le v els of abstrac tion implementat ion details ha ve less importan ce whereas they become dominant as the lev el of abstraction is lowered givi ng rise to practica l issues . Its goal is to find a contro lled transition among design lev els. 3 ASM f o r Grid Computing Before we define our m odel, we summarize the AS M for Grids defined in [16]. Figure 1 (a) sho ws the importan t eleme nts of th is m odel. The ASM univ erses of the model a re depicted on t he left o f the figur es, and on the right a graphical r eprese ntation of the connect ions of some elements of these univ erses and the most relev ant functio ns gov ernin g process execu tion are sho wn. In this model user application s consist of on e or more process es, while Grids c onsist of se ve ral nod es ha ving on e or m ore re source s. During the ex ecuti on of the user applicatio n first an agent maps the actual process of the applicatio n to a resource in the Grid, then the process is installed on the node of the resource as a task, which starts to use the resour ce. When all the pro cesses of the applicati on finish ed using their resource s, the application is finished. A. Ke rt ´ esz & Z s. N ´ emeth 21 (a) (b) Figure 1: (a) Basic elements of the ASM model for Grids, and (b) the modified ASM model. W e extend this formal model by introd ucing Grid brok ering at diff erent lev els. The basic model of grid systems introduced in [16] is present ed in a slightl y modified form here. T he modification is indi- cated by introducing more practical issues related to realization; align ing the model to the terminology and n aming con ventio ns of gri d brok ering ; and finally by e xper iences in Grid C omputing s ince th e pap er was publishe d. These m odificatio ns do not in val idate or alter the content and conclusi on of the initial model just add more relev ant details . The modificat ions are sho wn in Figure 1 (b). In the foll o wing subsec tions we define the basic elements of our propos ed ex tende d formal model based on ASM: the uni verses, the signature and the rules. 3.1 Univer ses and signature T o define our formal frame work, first w e need to examine real service Grid sys tems. Certain objects of the physical realit y are modeled as elements of univ erses and relations hips between real objects are repres ented as functions and relation s. In Grid systems users (uni ve rse U SE R ) define their applica tions in the form of jobs (uni ver se J OB ), w hich is the most typical computing paradigm for Grids hence, we restric t o ur model to this case. A job consist s o f one or more processes (uni vers e P R O CE SS ). The install ed instance s of processes are called as tasks (univ erse T ASK ), whi ch can be run on differe nt hosts (uni verse H OS T ). Hosts are the buil ding blocks of Grid systems, and typical ly a job is sent to a host for ex ecution. A host m ay ha ve sev eral nod es (e.g. w hen a host is a cluster), and nodes ha ve certain resour ces that processes require to run. Since nodes are usually in visible (and unmanag eable) for highe r le ve l too ls, ther efore we neglect t hem in o ur model. In this way one or more ph ysical re source s (uni ve rse PRE S O U R C E ) belong to a host, w hich also determines the physica l location (univ erse L O C AT I O N ) of the resourc es. The process es of jobs require some of these resources to run. Users should select a host accord ing to these resource requirements, which w e call as abstract resources (univ erse AR E SO U R C E ). Informat ion on the physical resourc es of the hosts can be gathered by querying the information system of a Grid. Once a job is submit ted to a host, it is map ped to physi cal resources during exec ution. While a resour ce is busy , the mapped process is in waiting state. When the resource becomes free, the process starts using it and enters running state. Process termination implies a done state in case of successful run, and a failed state in case of an error . In general, Grid authoriz ation allows users to log in to some 22 Formal Aspe cts of Grid Brokeri ng hosts and v alid ates user pri vileges to use some res ources of some hos ts [5]. The requested (abstr act) and the physic al resources hav e certain attribu tes (uni verse AT T R ). Compatibility between an abstrac t and a physical resource m eans the physical resource can satisfy the process require ment. Accord ing to this informal descrip tion, the follo wing functions are used in the m odel: job: PR O C E S S → J OB user , globaluser , localuser: J OB → U SE R submitte d: J OB × H OST → { t rue , f al se } procRequ est: PR O C E SS × ARE SO U R C E → { t r ue , f al se } uses: PR O C E S S × PRE SO U R C E → { t r ue , f al se } mapped: PRO C E SS → LO C AT I ON belong sT o: PRE SO U R C E × H O ST → { t rue , f al se } install ed: T AS K × LO C AT I O N → { t rue , f al se } attr: { ARE SO U R C E , PRE SO U R C E } → AT T R locatio n: PRE SO U R C E → LO CAT I ON handle r: PRE S O U R C E → PR O C E SS type: PRE SO U R C E → AT T R compatib le: AT T R × A T T R → { t rue , f al se } canLogin : U SE R × H O ST → { t rue , f al se } canUse: U SE R × PR E SO U R C E → { t r ue , f al se } jobState: J OB → { submi t t ed , r unning , wai t ing , d one , f ail ed } procState : PR O C E SS → { running , wait ing } e ven t: T ASK → { st art , abort , t er minat e } mappedHos t: J OB → H OST mappedRes ource: PR O C E S S × ARE SO U R C E → P RE S O U R C E 3.2 Initial state W e assu me that k processes belong to a job of a user . T he job an d its proc esses hav e some req uiremen ts, and no process and job is mapped to any resource or host. T herefo re the states of the jobs and processes are undefined . In the follo w ing we define the initial state of our model: ∃ p 1 , p 2 , . . . , p k ∈ PR O CE SS : j ob ( p i ) 6 = und e f , 1 ≤ i ≤ k ∀ p i , 1 ≤ i ≤ k : user( p i ) = u ∈ U SE R ∀ p i , 1 ≤ i ≤ k : ∃ ar ∈ ARE S O U R C E : procRequest( p i , ar ) = t r ue ∀ p i , 1 ≤ i ≤ k : ∃ pr ∈ PRE SO U R C E : uses( p i , pr ) = f al se ∀ j : mappedHos t( j ) = und e f ∀ p i , 1 ≤ i ≤ k : task( p i ) = und e f ∀ p i , 1 ≤ i ≤ k : mapped( p i ) = und e f ∀ j : jobState ( j ) = und e f ∀ p i , 1 ≤ i ≤ k : procState( p i ) = und e f ∀ u ∈ U SE R , ∃ pr 1 , pr 2 , . . . , pr m ∈ PRE S O U R C E : canUse( u , pr i ) = t r ue , 1 ≤ i ≤ m After w e ha ve defined the uni verses and the signature, in the followin g we giv e the rules o f our model A. Ke rt ´ esz & Zs. N ´ emeth 23 that consti tute a module, i.e . a progr am that is e x ecuted by eac h agent in the model. The model pre sented here is a distrib uted multi-agent AS M w here agents are jobs, i.e. eleme nts from the J O B univ erse. T he worki ng beha vior of the broker ing model is depicte d from the perspe cti v e o f the jobs hence, th e self functi on is represente d as j and means the iden tity of a job, i.e. it can ident ify itself among other agents. It is interpre ted differe ntly by dif ferent agents. 3.3 Rule 1: Resourc e selection Accordin g to F igure 1 (b), when the job is sent to a host, the required resource s need to be selected that are used by th e proce sses of the job . During job e x ecutio n, a task of each pr ocess is install ed to the loca- tion of the required and selected resource. T he preconditio n of resource selection is that the process of the job should be able to use the mapped resour ce. In case of the p rocess can d irectly acc ess the physical resour ce ( r d ) the ex ecutio n (resource usage) is automat ically started, otherwis e a local handl er process should provid e the exec ution platform (i.e. the additio nal software or service). If this hand ler proces s does not exist, it should be started before execu tion. T he agent responsibl e for resourc e mapping needs to ensure that the chosen resource fulfills the abstrac t resourc e requirement of the process. H ere is the formal definitio n: let h = m appedHos t( j ) let job( p ) = j let pr = mappedResou rce( p , ar ) if ( ∃ ar ∈ A RE S O U R C E ): procRequ est( p , ar ) = t rue & pr 6 = und e f & canUse(us er( p ), pr ) = t r ue then if type( pr ) = r d then mapped ( p ) : = location( pr ) install ed(tas k( p ), location( pr )) : = t r ue else if ( ¬∃ p ′ ∈ PR O CE SS ): handler( pr ) = p ′ extend PR O CE SS by p ′ with mapped ( p ′ ) : = locat ion( pr ) install ed(tas k( p ′ ), loca tion( pr )) : = t r ue handle r( pr ) : = p ′ do fo rall ar ∈ AR E SO U R C E procRequ est( p ′ , ar ) : = f al se enddo endextend endif procRequ est(p, ar) := false uses(p , pr) := true endif Π resource mapping if ( ∃ ar ∈ A RE S O U R C E , ∃ p ∈ PR O C E SS , ∃ h ∈ H OST ): job( p ) = j & mappedResou rce( p , ar ) = und e f & 24 Formal Aspe cts of Grid Brokeri ng procRequ est( p , ar ) = t rue & h = mapped Host( j ) then choose pr in PR E SO U R C E satisfy ing compatible(att r( ar ), attr( pr )) & belongsT o( pr , h ) = t r ue mappedr esourc e( p , ar ) : = pr endchoose endif Here, we note that though generally a job runs on a host (if it is a parallel job of communicating proces ses, it runs on a number of nodes of this host parallell y), some middle ware tools may enable co- alloca tion of parallel process es on nodes of dif ferent hosts. W e do not deal w ith this situatio n, since it is rarely used and suppo rted, but furth er refinement of our model could represent such cases. Before job exe cution it is necessary to authenticate users. In Service Grids users are authen ticated by prox ies of grid ce rtificates [5]. A local pro cess is respo nsible for v alidating these proxie s by mapping global users to local ones havin g the same pri vile ges. The related formalism of user m apping is similar to the one presente d in [16]. 3.4 Rule 2: State transition In this subsect ion we define, how job states are e v olving during ex ecuti on: if ( ∃ h ∈ H OST ): submitted( j , h ) = t r ue then jobState( j ) : = submitt ed endif if ( ∃ p ∈ PR O CE SS ): job( p ) = j & mapped( p ) 6 = und e f then procState( p ) : = w ai t ing jobState( j ) : = wait ing endif if ( ∃ pr ∈ PRE SO U R C E , ∃ p ∈ PR O C E S S ): job( p ) = j & uses( p , pr ) = t r ue then procState( p ) : = r unning jobState( j ) : = r unning endif if ( ∃ p ∈ PR O CE SS , ∃ t ∈ T ASK , ∃ pr ∈ PR E SO U R C E , ∃ h ∈ H OS T ): uses( p , pr ) = t r ue & bel ongsT o ( pr , h ) = t r ue & installe d( t , h ) = t r ue & ev ent( t ) = abort then jobState( j ) : = f ail ed PR O C E SS ( p ) : = f al se endif Though in genera l, process spawni ng could cause additio nal resource requests for job exec ution in a host, we do not detail this in our model, and keep it as abstract as possible, since at the lev el of grid brok ering proc ess communic ations and spawnin g are in visible. In order to handle these situations, we assume that resource requests of spawned process es are kno wn a priori. State transitio ns related to job terminati on are formalized in R ule 3. A. Ke rt ´ esz & Zs. N ´ emeth 25 3.5 Rule 3: T er mination Job ex ecuti on is terminated under the follo wing condition s: if ( ∃ p ∈ PR O CE SS ): job( p ) = j & procState ( p ) = r unning & ev ent(tas k( p )) = t erm ina t e then PR O C E S S ( p ) : = f al se endif if ( ∃ p 1 , . . . , p m ∈ PRO C E SS ): job( p i ) = j & jobState( j ) = f a il ed , 1 ≤ i ≤ m then PR O C E S S ( p i ) : = f al se endif if ( ¬∃ p ∈ P R O CE SS ): job( p ) = j & jobState( j ) = running then jobState( j ) : = d one endif 4 ASM f o r Grid Br okerin g This section focuses on middlew are components respon sible for broker ing in G rids. In our A SM model these components are represented by agents (al so ca lled as modules). First we g i ve an in formal ov ervi e w of these components and their roles in Grids and demons trate their usage in a typical G rid applica tion ex ecuti on scenario. In the follo wing subs ection s we sho w how these component s can appear as agents in the formal model describe d abov e. F urther more we emphasize how these bro ker ing componen ts con- trib ute to Grid Interoperab ility , i.e. ho w they suppor t transparen t job submissi ons to differ ent, separated Grids. At the lowest lev el of Grid resource manage ment we can find local resourc e managers (also called as sc hedule rs or clus ter managers, e.g. PBS [17]) that were tak en from h igh-pe rformanc e and distrib uted computin g, and now generally used in Grid Systems. Their goal is to schedul e and manage single and paralle l pro grams and their processes l ocally . This local management is formaliz ed in Rule 1 of or model. In this case interoperab ility is not supported at all. W ithout additional brokering component s users need to choose from the a v ailabl e hosts manually relying on mostly static infor mation. One le vel abo ve , a grid Res ource Manage ment System (RMS), a lso called a s a Grid resource brok er , is needed to automate host selection. It can be an internal middle ware service, o r an exte rnal tool that uses other middle ware components or services. (Note t hat the word ”resourc e” is used dif feren tly in our model as in the e xpression ”resou rce b rok er”. In o ur model we call the computing and s torage elements of Grid s as ”hosts”, and ”resources ” are t he physical co mponen ts of the hos ts, e.g. processo r and memory . ) While local managers usually schedule user programs in a grid host (e.g. in a cluster) among free processors , the Grid broker sche dules jobs at the lev el of Grid middle ware by selectin g a host that best matches the requ irements of these jobs . (Thus, the selected host can also be a clu ster with a loc al manager .) Therefore , a broker is also called as a meta-sch eduler – more informati on on broke r naming con ventions and their connecti ons can be found in [13]. S ome of them support differ ent middle ware solution s, job types, agreements or va rious quality of service (QoS) attrib utes. Furthermore differ ent broker s m ay be conne cted to dif ferent hosts and G rids. A taxonomy in [12] introdu ces these proper ties and shows the dif feren ces among the currently used brokers, their properties, organi zation and connection s and among their lev el of interopera bility . In our future work we also plan to represen t interoperabi lity as a metric in 26 Formal Aspe cts of Grid Brokeri ng our model in order to categ orize and dif ferent iate v arious brok ering components. W ith the help of grid brokers , host selectio n is automated, but users are still bound to separate grid island s ( i.e. grid systems that are complete systems on their own but closed to any form of interoperabi lity between ea ch o ther , either by technology , compatibility , administrati ve or othe r restric tions) managed by their own brok ers. Nev erthele ss users hav e the ability to select manually , which broker and Grid the y would like to use (ev en static informatio n on broke r propert ies are a v ailabl e in form of manuals or taxono mies e.g. in [12]). In orde r to achie ve the highest lev el of intero perabi lity broker selection should also be automated. T herefore at the high est le ve l we can find m eta-bro keri ng [1 1], w hich is a no v el approa ch that introduc es another layer abo ve curre nt grid broke rs in order to f acilita te inter -grid load balanc ing and interoperabl e broker ing. The Grid Meta-b rok er sits on top of Grid brok ers and uses meta- data to decide which broke r should be selecte d for a use r’ s job . T o demonstrate the interoperat ion of these brokeri ng components, w e describe a typica l Grid usage scenario for a job ex ecutio n that requires the follo wing steps: 1. The user defines its applic ation as jobs, also stating the requirements of its exec ution. 2. The user requir ements of the job is examined by the meta-bro ker , and mapped to the propertie s of the a vailab le brokers. A proper broker , that is able to submit the job, is selected for submission. 3. The s electe d broker exa mines the re source req uirement s of the job a nd matche s them to t he phys i- cal reso urces of the av ailable hosts. A host havi ng all the required resources is selected for exec u- tion. 4. The age nt on the s elected hos t (the local re source manager ) maps the res ource require ments of the job to the a v ailab le physical resources during exe cution . In the follo wing subsections w e define two mor e rules to model the informal descrip tion and discus- sion abov e. W e need addition al univ erse s and functio ns to incorpora te broke ring into our model. 4.1 An additional rule for Grid Broke ring Broke rs (univ erse BROK E R ) are responsib le for host selection, therefor e hosts are managed by brok ers, which can hav e dif ferent propertie s (uni verse PR OR E RT Y ) tha t users m ay requi re for job execu tion. A user should select a broker for its job according to these requiremen ts (uni verse RE Q U I R E M E N T ). Furthermor e we place uni vers e ARE SO U R C E as a subset of univ erse RE Q U I RE M E N T , since the ele- ments of both sets represent user requirement s, and univ erse PRE SO U R C E can be a subset of uni ver se PR ORE RT Y , b ecause physic al resou rces can be reg arded as host p ropert ies. The follo wing functio ns are added to the model: reques t: J OB × RE Q U I RE M E N T → { t r ue , f al se } submitte d: J OB × { H OST , BR OK E R } → { t r ue , f al se } manages : H OS T × BR O K E R → { t r ue , f al se } ha ve : B R O K E R × PR ORE RTY → { t r ue , f al se } attr: { RE Q U I R E M E N T , PR OR E RTY } → AT T R W e exten d the initial state by: ∀ j : ∃ r ∈ RE Q U I RE M E N T : reques t( j , r ) = t r ue A. Ke rt ´ esz & Zs. N ´ emeth 27 (a) (b) Figure 2: (a) Grid brok ering and (b) Meta-brok ering in the ASM model. W e exten d Rule 2 with the follo wing state changes: if ( ∃ b ∈ B R O K E R ): submitted( j , b ) = t r ue then jobState( j ) : = submitt ed endif if ( ∃ h ∈ H OST ): submitted( j , h ) = t r ue then jobState( j ) : = wait ing endif Once a broker is selected by the user , it should find an execu tion host. The preconditio n of this host selecti on process is that the user of the job should be able to use the require d resources of the selected host. The broker agent responsibl e for host mapping needs to ensure that the chosen host has all the re- source s requested by the process es of the job . This additi onal component responsib le for Grid broke ring is highligh ted in Figure 2 (a). In the follo wing w e state the formal definitio n: Rule 4: Host selection h = mappedHo st( j ) if ( ∃ ar 1 , . . . , ar m ∈ AR E SO U R C E , ∃ pr 1 , . . . , pr m ∈ PRE S O U R C E ): reques t( j , ar i ) = t r ue & h 6 = und e f & canUse(user( j ), pr k ) = t r ue , belongsT o( pr k , h ) = t r ue , 1 ≤ i , k ≤ m then submitted ( j , h ) : = t r ue endif Π host mapping if ( ∃ j ∈ J OB , ∃ ar 1 , . . . , ar m ∈ ARE S O U R C E , ∃ pr 1 , . . . , pr m ∈ PR E SO U R C E ): 28 Formal Aspe cts of Grid Brokeri ng mappedHos t( j ) = und e f & request( j , ar i ) = t r ue , 1 ≤ i ≤ m then choose h in H OS T satisfy ing compatible(at tr( ar i ), attr( pr k )) where b elong sT o( pr k , h ) = t r ue , 1 ≤ i , k ≤ m mappedh ost( j ) : = h endchoose endif 4.2 Rule 5: Br oker selection At the highest le vel of Grid resource management a broker needs to be selecte d automatically for a user job . An importan t precond ition of the selection process is that such a brok er needs to be selected that manages hosts with resources that the user of the job can use. Furthermore the agent resp onsibl e for brok er selection, the meta-broke r (uni verse M E T ABR OK E R ) needs to ensure that the chosen broker has all the properties required by the user’ s job . Ther efore users need to characterize their job requiremen ts in a certain job descripti on language, which should include both the required broker proper ties and ab- stract resources of the process es of the job . This additio nal Grid m iddle ware component is highlight ed in Figure 2 (b). The follo wing function is added to the model: mappedBro ker: J OB → BROK E R W e exten d the initial state by: ∀ j : mappedBro ker ( j ) = und e f The formal definitio n of the meta-broke r agent is as follo ws: let b = mappedBro ker( j ) if ( ∃ r ∈ RE Q U I RE M E N T , ∃ pr ∈ PRE SO U R C E , ∃ h ∈ H OS T ): reques t( j , r ) = t r ue & b 6 = und e f & canUse(u ser( j ), pr ) = t r ue , belong sT o( pr , h ) = t rue , m anages ( h , b ) = t r ue then submitted ( j , b ) : = t r ue endif Π broke r mapping if ( ∃ r 1 , . . . , r m ∈ RE Q U I RE M E N T , ∃ p 1 , . . . , p m ∈ PR O PE R T Y , ∃ j ∈ J OB , ∃ b ∈ B R O K E R ): mappedBro ker( j ) = und e f & ∀ i : request( j , r i ) = t r ue & ∀ i : hav e( b , p i ) = t r ue , 1 ≤ i ≤ m then choose b in BR OK E R satisfy ing compatible(at tr( r i ), attr( p i )), 1 ≤ i ≤ m endif A. Ke rt ´ esz & Zs. N ´ emeth 29 Finally w e should state that jobs can be interconne cted in order to form a complex G rid application called as workflo w s. The exe cution of workflo ws require a coordinat ing tool called w orkflo w en actor that schedu les the interd epende nt jobs for execu tions. W e refrain from formalizing workflo w managemen t and incorpora te it into ou r model, since ou r central ent ities are jo bs, and th erefor e assu me th at grid applic ations are submitted into the system in the form of jobs. As a summary , we hav e sho wn that grid brokerin g tak es place at three lev els, and the follo wing operat ions need to be per formed: brok er mapping , host map ping an d res ource mappin g. In the follo wing sectio n we sho w , ho w practical e xamples of these compon ents can be describe d by our formal ASM model with the help of ASM refinement. These tools are the Grid Meta-Broke r [11] and GT brok er [10]. 5 Refinement of Br oker Componen ts This section contains illustrati ve examples, ho w the generic brokerin g model can be refined into models that represent realised implementations of the brokeri ng principles. One can see in these exampl es how certain functions, kept abstract in Rule 4 and 5 presented earlier , are transfo rmed to re vea l implementa- tion details. More informa tion on the realization and practical utilizatio n of these tools can be read in [10]. 5.1 Refinement of br oker mapping (matchmaking of the Grid Meta-Br oker) Π ′ broke r mapping if ( ∃ r 1 , . . . , r m ∈ RE Q U I RE M E N T , ∃ p 1 , . . . , p n ∈ PROPE RT Y , ∃ j ∈ J OB , ∃ b 1 , . . . , b l ∈ BR O K E R , ∃ v 1 , . . . , v l ∈ RE AL ): mappedb rok er( j ) = und e f & ∀ t : reques t( j , r t ) = t r ue , 1 ≤ t ≤ m , & ∀ i : h a ve( b k , p i ) = t r ue , 1 ≤ i ≤ n , 1 ≤ k ≤ l then do f orall k ( 1 ≤ k ≤ l ) v k : = getBrokerPer f( b k ) if ( ¬∃ t , i ): attr( r t ) = attr( p i ) & ha ve( b k , p i ) = t r ue , 1 ≤ t ≤ m , 1 ≤ i ≤ n then v k : = 0 enddo choose v max in ( v 1 , . . . , v l ) satisfy ing v max ≥ v k , 1 ≤ k , m ax ≤ l mappedb rok er( j ) : = b max endchoose endif In ad dition t o the broker mapp ing define d in Rule 1, this refinement d etails ho w th e comp atible func - tion is implemented. In case of the Grid M eta-bro ker , the attrib utes of the brok er propertie s are certain ke ywo rds. The users ha ve to use the same keyw ords in their require ment specificati ons, therefo re com- patibi lity means e xact string ma tching . T he refined agent a lso u ses a n add itiona l function g etBrok erPerf: BR OK E R → RE AL , which returns a real number denoting the dynamic performanc e of the appropriate brok er . The hig her this valu e is the better the broke r performs. 30 Formal Aspe cts of Grid Brokeri ng 5.2 Refinement of host mapping (matchmaking of GTbr oker) Π ′ host mapping if ( ∃ j ∈ J OB , ∃ ar 1 , . . . , ar n ∈ AR E SO U R C E , ∃ pol icy ∈ R E Q U I RE M E N T , ∃ pr 1 , . . . , pr m ∈ PRE S O U R C E , ∃ h 1 , . . . , h t ∈ H OST , ∃ r 1 , . . . , r t ∈ RE AL ): mappedh ost( j ) = und e f & request( j , pol icy ) = t r ue , reques t( j , ar i ) = t r ue , 1 ≤ i ≤ n ≤ m then do f orall k ( 1 ≤ k ≤ t ) r k : = countRank( pol icy , h k ) if ( ¬∃ l , i ): attr( ar i ) ≤ attr( pr i ) & belon gsT o( pr l , h k ) = t r ue , 1 ≤ i ≤ n , 1 ≤ l ≤ m then r k : = 0 enddo choose r max in ( r 1 , . . . , r t ) satisfy ing r max ≥ r k , 1 ≤ k , m ax ≤ t mappedh ost( j ) : = h max endchoose endif In additi on to the host mapping defined in Rule 2, this refinement also rev eals the meaning of the compatib le function. In case of G Tbrok er , the attrib utes of resourc e require ments denote the amount of resour ce ca pacity (e.g. memory size o r process or speed ) needed by the processes of the job for ex ecution . This means, if the av ailable physical resource has equal or greater capacity than requeste d, the process can run. The host selection method can be influenc ed by users using the special pol icy requ irement. The v alue of its attribu te tells the addition al countRank : RE Q U I RE M E N T × H OS T → RE AL function ho w to compute the rank for the a v ailable hos ts (e.g. higher priorit y can be giv en to hosts w ith faster proces sors). Finally , the host with the highest rank is selected for exe cution . 6 Conclusions In thi s pape r we ha ve in vest igated the bro ker ing componen ts of the Grid middle ware an d defined a three-l ayered formal model using Abstract State Machines . In this model three agents are responsibl e for resourc e management by performing three selec tion processes: broker mapping, host mapping and resour ce mapping . W e hav e also proposed two refi ned definitions for broker and host selection, which are implemented by the Grid Meta-brok er and GT brok er . Our future work aims at introd ucing inter - operab ility metrics for categori zing brokerin g componen ts and using the ASM model for verify ing our cate goriza tion. Refer ences [1] Alten hofen M., A. Friesen and J. Lemcke (2008) : ASMs in S ervice Oriented Ar chitectures . Journal o f Universal Com puter Science , vol. 14, no. 12, pp. 2034 –2058 . [2] B ¨ orger , E., and R. Stark (2003 ): Abstract Sta te Machines. A method for High-level System Design and Analysis . Springer . A. Ke rt ´ esz & Zs. N ´ emeth 31 [3] B ¨ orger , E. and B. T halheim (2008 ): Mod eling W orkflows, In teraction P atterns, W eb Services and Bu siness Pr ocesses: Th e AS M-Based Appr oach . In Procee dings of the 1st internation al Conference on Abstract State Machines, B and Z, Lecture Notes In Computer Science , vol. 5 238. Springer-V erlag, pp . 24–38 . [4] Brato sin, C., W . Aalst, N. Sidorova, and N. Trcka ( 2008 ): A R efer ence Model fo r Grid A r chitectures and Its Ana lysis . In Pro ceedings of th e OTM 20 08 Con federate d intern ational Conferen ces, Lectur e Notes In Computer Science , vol. 5331 . Springer-V erlag, pp . 898–9 13. [5] Foster, I., C. Kesselman, G. Tsudik, a nd S. T uecke (19 98): A secu rity a r chitectur e fo r comp utationa l grids . In Proceedin gs of the 5th A CM Conference on Computer and Communicatio ns Security , A CM, pp. 83 – 92. [6] g Lite middleware f or grid computing , A vailable at http://glite.web.cern.ch/glite/ . [7] Glo bus T oolkit, A vailable at http://www.globu s. org/toolki t/ . [8] Gu revich, Y . (1993) : Evolving Algebras: A n A ttempt to Discover Semantics . Current T rends in Th eoretical Computer Science , pp. 266–292 . [9] Gu revich, Y . (1 997): Draft of the A SM Gu ide , A vailable at h ttp:/ /www.eecs.umich. edu/gasm /papers/ guide9 7.html . [10] Kert ´ esz, A., an d P . Ka csuk (2 009): Grid Inter operability S olutions in Grid Resource Management . IEEE Systems Journa l , vol. 3, Special Issue on Grid Resource Management, pp. 131–141 . [11] Kert ´ esz, A., and P . Kacsuk (2009 ): GMBS : A new mid dlewar e s ervice for ma king grids inter op erable . Future Generation Computer Systems , doi:10 .1016 /j.future .2009.10.007. [12] Kert ´ esz, A., and P . Kac suk (2007 ): A T axo nomy of Grid Resou r ce Br okers . 6th Austrian-Hu ngarian W o rkshop on Distributed and Parallel Systems (D APSYS 2006) , Springer US, pp. 201– 210. [13] Kert ´ esz, A., an d T . Pr okosch (2010): Th e Anato my of Grid Resour ce Management . Remote Instrumen tation and V irtual Labor atories, Service Architectur e and Networking , Springer US, to be appear . [14] Kes selman, C., and I. Foster (199 8): T he Grid: Blueprint for a New Compu ting In frastructur e . Morgan Kaufman n Publishers . [15] Krauter, K. , R. Bu yya, an d M. Mah eswaran (2002) : A taxo nomy and survey of grid r esour c e management systems for distributed computing . Softw ., Pract. Exper . , v ol. 32, pp. 135–164 . [16] N ´ emeth, Zs., and V . Sunder am (200 3): Characterizing Grids: Attributes, Defin itions, and F ormalisms . Journal of Grid Computing , vol. 1, pp. 9–23. [17] PBS GridW orks, A vailable at http://www.pbsgridw orks. com/ . [18] Uniform Interface to Computing Resources (UNICORE), A vailable at http:// www.unic ore.eu/ .