MultiKulti Algorithm: Migrating the Most Different Genotypes in an Island Model

MultiKulti Algorithm: Migrating the Most Differen t Genot yp es in an Island Mo del Lourdes Araujo 1 , Juan J. Merelo Guerv´ os 2 , Ca rlos Cotta 3 and F rancisco F ern´ andez de V ega 4 1 Dpto. Lengua jes y Sistema s Inform´ aticos. UNED, Madrid 28040, Spain, lurdes@lsi .uned.es 2 Departamento de Arquitectura y T ecnolog ´ ıa de Computadores, U niv ersidad de Granada, Granada 18071 , S pain, jmerelo@ geneura.ugr .es , 3 Departamento de Lengua jes y Ciencias de la Computaci´ on, Universidad de M´ alaga, Spain, ccottap@lc c.uma.es , 4 Departamento de Arquitectura y T ecnolog ´ ıa de Computadores, U niversidad de Extremadura, Spain, fcofdez@un ex.es Abstract. Migration p olicies in distributed ev olutionary al gorithms has not b een an activ e researc h area until re cently . Ho w ever, in t he same w a y as op erators hav e an impact on p erformance, the c hoice of migran ts is due to hav e an imp act to o. In this pap er we prop ose a new p olicy (named multikulti ) for choosing th e individuals that are going to b e sen t to other nod es, based on mul ticultur ality : the individu al sen t should b e as different as possible to the receiving population (represen ted in several p ossible wa ys). W e hav e c h ecked this policy on different discrete opti- mization problems, and found that, in ave rage or in median, this p olicy outp erforms classic al ones like sending the b est or a random individual. 1 In tro duction and state of t he art Evolutionary alg orithms (EAs) make individuals in the p opula tio n evolv e in parallel, whic h suggests that the exploitation of par allelism ca n be quite natu- ral in these algorithms. This ha s led to ma ny efforts to par allelize them with the inten tion of reducing the execution time a nd improving the qua lit y of the solutions. There hav e b een several attempts to clas s ify the numerous works de- voted to parallel EAs [1,2], with island or c o arse gr aine d mo dels b eing one of the most p opular a pproach of par a llelization of EAs. This a pproach is usually implemen ted on distributed systems, since it does not require a high cos t in com- m unications: no des only exchange a few individuals after several g enerations. The p opula tion is divided in subp opulations, whic h usua lly evolve isolatedly ex- cept for migr ations , that is, the exchange o f some individuals a fter a num b er of generations . The b ehavior of the isla nd mo del differs from the sequential EA, since the co mp os ition, and thus the dyna mics, of every subp opula tion is differ- ent : since subpopulatio ns are smaller than the whole po pulation, the parallel EA will conv erge faster. F urthermor e, migrations amo ng subpopula tions usually im- prov e the quality of the sequential solutions [3], which makes the pa rallel mo del 2 int eresting even for sequential executions. No t only do es this apply to g e netic algorithms; similar r esults have b e e n found for Genetic Progr amming: T omassini et a l. [4] hav e analyzed diversit y in multipopulation genetic pr ogra mming (GP) finding a correla tion b etw een diversity and the b etter convergence pro pe rties of distributed GP . These re s ults hav e inspired this work. Diversit y in the subp opulatio n is so impo rtant that it leads to improvemen t in quality a nd efficiency at the sa me time. Accordingly , we have lo oked for the wa y o f enhancing the div ersity induced by the usual migr ation p olicies, by using the no tio n of mu lticultur ality : mig r ants should be chosen on the bas is of genot ypic differe nc e to the r eceiving population; we have called this new migr ant-selection policy mu ltikulti 1 . Let us then loo k at ho w these diversit y enhancement could be rea lized through migration p olicies , which include several issues: – the num ber of individuals undergo ing migration, – the frequency of migra tion, i.e. the n umber of genera tions or ev alua tions betw een migrations, – the p olicy for selecting migrants, – the migr ation re placement p olicy , – the top olo gy of the c ommunication among subp opulations, – the synchronous or asynchronous nature of the communications. These issues have b een inv es tigated in different pap er s: Alba et al. [5] c o mpare synchronous and async hronous migra tion p olicies, Herrera et al. [6 ] studied some of the aforementioned issues in a hier archical configuration of subp opula tions, and C a nt´ u-Paz [7,8], Alba and T roy a [9], and No da et a l. [1 0] have analy zed different migra tion p olicies. Se veral results presen ted in these mentioned works indicate tha t diversity is a fundamen tal key in the success of the island mo del. F or example, works co mpa ring synchronous versus asynchronous mo dels [5], have found that the a synchronous alg orithms outp erformed the synchronous ones in all the exp eriments. Cant´ u-Paz [7,8] has studied the four p oss ible combinations of r andom and fitness-based emig ration a nd re placement of existing individuals. He found that the migra tion p olicy that causes the greatest reduction in work (takeo ver time 2 ) is to ch o ose b oth the migrants a nd the replacements according to their fitness, b ecause this p olicy increa ses the selection pressure and may ca use the algor ithm to c onv er ge s ignificantly faster . Ho wev er , if co nv er g ence is to o fa st it can lead to algorithm failure, as Cant´ u-Paz [8] states referr ing to parallel EAs: R apid c onver genc e is desir able, but an exc essively fast c onver genc e may c ause the EA to c onver ge pr ematur ely to a su b optimal solution. So, other p olicies mu st also b e consider e d. In fact, Alba a nd T r oy a [9] found that migration o f a r andom string prev ent s the “ conquest” effect in the target 1 Multikulti, as defined b y the wikipedia entry ( http://en. wikipedia.o rg/wiki/Multikulti ), is an slogan for a m ulticultural approac h to public policy 2 Number of generations required to conve rge to the b est individual from t h e initial p opulation, by applying selection only 3 island for small or medium size d sub-populations . Noda et al. [10] ha ve prop osed choosing whic h individua ls to mig rate and/or repla ce a daptively dep ending on some knowledge-oriented rules . T o do this, each agent rec eives informa tio n ab out the fitness function from its peer s. The tested adaptive po licies have prov ed useful providing best solutions than the sequential execution. In spite of the results shown above, there is still a num b er of issues that have not b een inv estigated yet. In this work we also fo cus on the p olicy for migr ants selection. Previous works dealing with this asp ect have studied the use o f any of the se le c tion op era tors usually applied in evolutionary a lgorithms: prop ortiona l selection, tour nament, ra ndom, etc. Only the w ork by Noda et al. [10] consider s, among other po licies, one in whic h the individuals sent ar e c hosen to be quite different from other s previo usly sent. The aim of this w ork is to explo it differences in the v a rious subp opula tio n. T o do this, we focus on the selection of the individuals to b e s e n t to other subp opula- tion. Our thesis is that migrating individuals differen t enough to the destination subpo pulation instead o f the be s t individuals can result in a better p erforma nce through the diversit y enha nc e men t it pro duces. Consider the Figure 1 with t wo subpo pulations. The bla ck po ints repr esent the distribution o f the p opulation along the function to optimize. Individual a in subpopulatio n P1 has the high- est fitness, and thus it would b e sent to subp opula tio n P 2 following the most common migr ation p olicie s. W e prop os e to send individual b , whose genotype is quite different from those of subpopulation P2. In the example it would lead to explor ation of a ne w area of the s e arch space where the globa l optimum is placed. In order to achieve this, the pro c e s s corres po nding to subp opulatio n P1 needs to r eceive information on the compositio n of the individuals in subp opu- lation P2. W e ha ve co nsidered different ways of providing this informatio n in a concise manner. One of them is taken the b est individual of s ubpo pulation P2 as repres ent ative. The other one is using a kind of average genotype, the con- sensus sequence describ ed la ter, as repres ent ative of subp opula tion P 2. Another impo rtant issue to inv estigate if the trade-off b etw een promoting div ersity a nd fav or ing the bes t individuals. The risk of sending the most differen t individual as migrant is that if its fitness v alue is low compar ed to thos e o f the de s tination subpo pulation, the migr ant would probably disapp ear immediately . Therefore, another question tested in the ex p er iment s has b een if the most different indi- vidual is fit e nough to survive when migrating, or if it is b est to select the most different from an elite. The r est of the paper pro cee ds a s follows: section 2 describes the mo de l de- tails; section 3 is dev oted to describe the evolutionary algor ithm and its imple- men tation; section 4 present s and dis cusses the exper imen tal results, and sec tio n 5 draws the main c o nclusions of this work. 2 Mo del Descript ion W e have co nsidered a r ing top ology (Figur e 2), in whic h each node can send o ne or more individuals to the next no de in the r ing. T o per form the c hoic e of the 4 P2 P1 a b Fig. 1. Illustra tion of the mu ltikulti migr ant selection p olicy: Individual a in subpo pulation P1 has a hig her fitness than individua l b . Howev er b is more different to the best individual in subp opulation P2. Accordingly , b is s e le cted to be sent in the migr ation. P1 P2 P3 PN representation G2 genotype "quite" different from G2 "quite" different genotype representation G3 from G3 Fig. 2. Scheme of the multikulti algo rithms. migrants, the no de P i receives from no de P i +1 information ab out the genot ype of its subpo pulation. W e hav e consider ed tw o differen t wa ys of representing this information in a concise manner: 1. With the bes t individual o f the subp opulation. After a n umber of gener ations without exchanging individuals we ca n exp ect that each subp opula tion is close eno ugh to conv e rge for the bes t individual being a fair re pr esentation of the whole p opulation. 2. With the conse ns us seq uence of the p opula tion. This is a co ncept tak en fr om biology where it is defined as the sequence that r eflects the most common choice of base or amino acid at each p osition of a geno me. Are a s of par- ticularly g o o d agre e ment often repres e nt conserved functional domains. In our case it is comp osed of the most frequent alelo for each pos ition of the genotype. Once the no de P i has got this infor mation, it sends to no de P i +1 an individ- ual different eno ugh from the subpo pulation P i +1 representative. Her e w e hav e considered t wo a ppr oaches: – Base : Selecting the most different from the subp opulation P i . – Elite : Sele c ting the most different among the b est half of subp opulation P i . 5 3 Implemen tation Chromoso mes [1 1] of our GA are fixe d- length binar y strings . The selectio n mec h- anism to choo se individuals for the new po pulation uses a steady state alg orithm, with t wo-point cr ossov er op er ator and single-bit-flip m utation. The rest of the parameters a re shown in ta bles 1(a) and 1(b). P arameter V alue P opulation 32 Selection rate 60% Generations to migration 20 Mutation priorit y 2 2-p oint crosso ver priority 3 P arameter V alue Chromosome length 120 P opulation 256 Selection rate 20% Generations to migration 20 Mutation priorit y 2 2-p oint crosso ver priority 3 Max num b er of ev aluations 200000 (a) (b) T able 1. E volutionary algorithm parameter s used in the P -Peaks (a) and in the MMDP (b) exp eriments. P-Peaks a nd the massively m ultimo da l deceptive problem ( MMDP ), tw o of the three disc rete optimizatio n problems pres ented b y Giaco bini et al. in [12] hav e b een selected for tes ting . These problems , while b eing both m ultimoda l, represent differen t degrees o f difficult y for parallel evolutionary optimization. They will b e describ ed b elow. These tw o problems hav e been implemented and in tegrated in the Algo- rithm: :Evol utionary librar y , whic h is freely av aila ble under the GPL license from h ttp:/ /opeal .cvs.sourceforge.net/opeal/Algorithm- Evolutionary/ . In o r der to sim ulate a paralle l algorithm, the c o op er ative multitasking P erl mo d- ule POE has been used; each node is represe n ted b y a P O E session . The r est of the evolutionary alg o rithm has b een implement ed using the same Algo - rithm: :Evol utionary P erl mo dule [13]. T he program, alo ng with the parameter sets used, is also av ailable under a n op en source licens e from the same site. In this sim ulated parallel scenario , each no de runs a rank-based substitution, steady state a lgorithm. At the end of a preset num b er of g enerations, each node sends a sing le individual to the other no de a ccording to the p olicy being tes ted 3.1 Problems tested Two functions hav e b een us e d for testing: P-Peaks and MMDP , tw o of the three discrete optimization problems presented in [12]: The mass ively mult imo dal de- ceptive problem ( MMDP ) and the problem gene r ator P-Peaks . These problems, while b eing bo th multimodal, represent differen t degrees o f difficult y for parallel evolutionary optimization. They will b e des crib ed next. 6 The MMDP [14] is a deceptiv e problem compose of k subpr oblems of 6 bits each one ( s i ). Dep ending of the n um b er of ones (unitation) s i takes the v a lues depicted nex t: f itness s i (0) = 1 . 0, f i tness s i (1) = 0 . 0, f i tness s i (2) = 0 . 3603 84, f itness s i (3) = 0 . 6405 76 f i tness s i (4) = 0 . 360384 , f i tness s i (5) = 0 . 0, f itness s i (6) = 1 . 0 The fitness v alue is defined as the summatory of the s i subproblems with an optimum of k (equation 1). The num ber o f lo cal optima is quite large (22 k ), while there are only 2 k global solutions. In this pa p e r , we consider a single instance with k = 20. f M M DP ( s ) = k X i =1 f itness s i (1) The P-Peaks pr oblems is a m ultimo dal pro blem generator prop os ed b y De Jong in [15], and is created by genera ting P random N − bit strings where the fitness v alue of a string x is the num b er of bits that x has in c ommon with the nearest pea k divided by N . In the ex per iments made in this pap er w e will cons ider P = 100; the optimum fitness is 1 .0. f P − P eaks ( x ) = 1 N max 1 ≤ i ≤ p { N − H amming D istance ( x , P ea k i ) } (2) W e consider an instance of P = 100 and 64 bits where the optimum fitness is 1.0 (E q uation 2). These t wo pro blems hav e b een also implemen ted and in tegrated in the Al- gorith m::Ev olutionary library . 4 Exp erimen tal results First, w e tested several parameter co nfigurations for the P-P eaks problem. In general, when div ersity conditions are not to o harsh, the perfo r mance difference betw een different migration p olicie s is not to o high. Eventu ally , when the going gets tough, differences such as those shown in Figure 3 do app ear . In that graph, taken for a 8-no de, p opula tion = 32 exp eriment, results are quite differen t de- pending on the migr ation po licy . F or starters, sending the b est individual yie lds the w orst results. If we cons ider the media n, the multikulti is similar to the r an- dom p olic y , but its behavior is b etter if we consider the a verage and the worst case, a s shown in table 2. The MMDP was also tested with a similar setup; results are shown in figure 4 where three new versions of the algo rithm, named multikulti -elite , consensus- multikulti and multiku lti-elite-co nsensus , ha ve b een tested. Multikulti-el ite c ho o ses the individual most different to the receiving popula tio n best individual, but only among the 50% b est. In this ca s e, no t sur prisingly , this strateg y b eats the m ultikulti by far as well a s the random stra tegy , but more closely . This is proba- bly due to the natur e of the MMDP : it is a deceptive problem, where increasing diversit y mig ht not have the desir ed result, since co mpe ting conv entions for the 7 best mk random 1000 2000 5000 10000 20000 50000 Policies comparison, P−Peaks problem type evaluations Fig. 3. Boxplot (with logarithmic y axis) of the n umber o f ev aluatio ns needed to find the solution in the P-Peaks proble m. b est repres ents the b ehavior o f the exp eriment whe n the b est individual in the p o pula tion is sent, r andom with a random individual, and finally mk , in the middle, s tands for multikulti , the algo- rithm we a re testing in this pap er which sends the individua l in the p opulation most different to the b est in the receiving p opulation. same 6-bits p ortion can lead in ea ch p opulatio n. This wh y , instead of s ending the most differen t as the multikulti p o licy does , doing it w ith o ne that is different enough, but at the same time, fit enough, pro duces the b est results. The second stra tegy tested, consensus-mu ltikulti , achiev es an intermediate per formance among the t wo, b eating the multikulti, but achieving worse res ults than mu ltikulti-el ite . This stra tegy sends the individual that is most different to the co nsensus string, that is, the str ing whose every bit represents the ma jor it y v alue for that p o s ition among the p opula tion. The result is pro bably due to the same reason: a v alue that is to o different represents a high disturbance, and th us is bad for diversit y . In fact, the third strategy , multik ulti-elite-c onsensus , which sends the individual most different to the cons ensus string which is among the 8 P olicy Median Average Best 25820 25310 Random 1545 6410 Multikulti 1544 1252 T able 2. Statistics for num b er o f ev a luations of different migra tion p olicies for the P-Peaks problem. 50% best, achiev es results similar to Multik ulti-elite , althoug h, in ge ne r al, a bit better . In order to in v estigate what is g oing on, w e ha ve measured the en tr opy for the MMDP . The results ar e s hown in Figur e 5, which s hows the differe n t evolution paths of pheno typic entropy (computed using the Shannon fo r mula) with the m ultikulti-elite migration p olicy (left) and the b e s t migr ation p olicy (center). The behavior es quite different. The m ultikulti p olic y , not only k eeps the en tr opy high, but cons iderably increases it in s o me po pulations during evolution. The po licy of migrating the b est provides quite muc h low er levels of entropy; with a decreasing trend tha t never c hanges, leading to a collapse of entropy from cycle 12. This pr oves the utilit y of the multikulti p olicy to main tain div ersity , and suppo rts the result tha t the improvemen t in the num ber of e v aluations is due precisely to this diversit y-enhancing effect broug h b y the mu ltikulti p olicies. 5 Conclusions This paper ha s ex plored new a lternatives to promote diversity in an island model. This is achieved b y s e lecting as migrant individuals with a g enotype different enough to the destination p opulation. Because ther e is a trade - off be tw een pro - moting diversity and fa voring the b est individuals, we hav e pe r formed ex per i- men ts to find out the degree of differ e nc e which pro duces the b est results. These exp eriments have shown that results only improv e substa ntially if the migrant is chosen from the elite; that is, those with an a b ov e- av er age fitness, which in- dicates that diversity only improves the results if the incoming mig rant has a minim um lev e l of qualit y . W e hav e compared tw o different wa ys of c haracterizing the destination p opulation: the best individual and the c onsensus sequence. Re- sults hav e shown that both of them represent appro priately the p opulation, with the consensus sequence perfor ming o nly sligh tly better. Studying the phenotypic ent ropy of the po pulation w e ha ve found that our metho d effectively impr ov es ent ropy by av oiding entrop y to fall to o fast and also creating an entropy differ - ent ial a mo ng p opulatio ns , and thus div ersity . In the future w e in tend to develop a par a llel implemen tation of the system, which will allow us to measur e also execution times. W e are a lso working on al- ternative mechanisms to characterize the des tination po pulation, and th us select the more appropriate migrants. W e will als o test results obtained b y c hanging other algo r ithm parameter such as num b e r of migra nts or the num ber of no des. 9 Ac kno wledgemen ts This pap er has b een funded in part by the Spanish MICYT pro ject NoHNES (Spanish Ministerio de E ducaci´ on y Ciencia - TIN20 07-68 083) and the Junta de Andaluc ´ ıa P06- T IC - 0202 5 . References 1. Can t ´ u-Pa z, E.: Efficient and A ccurate Para llel Genetic Algorithms. Kluw er Aca- demic Publishers, N orw ell, MA, U SA (2000) 2. Konfrst, Z.: P arallel genetic algorithms: Adv ances, computing trends, applications and p ersp ectives . In: IPDPS. (2004) 162ff http://csd l2.computer .org/dl/proceedings/ipdps/2004/2132/07/213270162b.pdf . 3. Alba, E., Nebro, A., T roy a, J.: Heterogeneous computing and parallel genetic algorithms. Journal of Pa rallel and Distributed Computing 62 (9) (2002) 1362– 1385 4. T omassini, M., V anneschi, L., F ernandez, F., Galeano, G.: A study of diversi ty in multipopulation genetic programming. In Lia rdet, P ., Coll et, P ., F onlup t, C., Lut- ton, E., Schoenauer, M., eds.: Evolution Artificielle, 6th International Confere nce. V olume 2936 of Lecture Notes in Computer Science., Springer (2003) 243–255 5. Alba, E., T roy a, J.M.: Analyzing synchronous and asynchronous parallel dis- tributed genetic algorithms. F uture Generation Comp. Sy st. 17 (4) (2001) 451–465 6. Herrera, F., Lozano, M., Moraga, C.: Hierarc hical distributed genetic algorithms. International journal of intell igen t systems 14 (11) (1999) 1099–11 21 7. Can t ´ u-Pa z, E.: Migration p olicies and takeo ver times in genetic algorithms. I n : GECCO. (1999) 775 8. Can t ´ u-Pa z, E.: Migration p olicies, selection p ressure, and parallel evolutionary algorithms. Journal of Heuristics 7 (4) (2001) 311–334 9. Alba, E., T roya , J.M.: I nfluence of th e migration p olicy in parallel d istributed gas with structured and p an mictic p opulations. Ap pl. Intell. 12 (3) (2000) 163–181 10. Noda, E., Coelho, A., Ricarte, I., Y amak ami, A., F reitas, A.: D evising adaptiv e migration p olicies for coop erative d istributed genetic algorithms. Systems, Man and Cyb ernetics, 2002 IEEE International Conference on 6 (6-9 Oct. 2002) 6 pp . vol .6– 11. Holland, J.J.: Adaptation in Natural and Art ificial Sy stems. Universit y of Mic higan Press (1975) 12. Giacobini, M., Preuss, M., T omassini, M.: Effects of scale-free and small-w orld top ologies on binary co ded self-adaptive CEA. In Gottlieb, J., Raidl, G.R., eds.: Evol utionary Computation in Com binatorial Optimization – EvoCOP 2006. V ol- ume 3906 of LNCS., Budap est, Springer V erlag (2006) 85–96 13. Merelo-Guerv´ os, J.J. : Evolutionary computation in Perl. In Perl Mongers, M., ed.: Y APC::Europ e::2002. (2002) 2–22 14. Goldberg, D.E., Deb, K., Horn, J.: Massiv e multimodality , d eception, and genetic algorithms. In R. M¨ anner, Manderic k, B., ed s.: Para llel Problem Solving from Nature, 2, Amsterdam, Elsevier Science Publishers, B. V. (1992) 15. Jong, K.A.D., Potter, M.A., Sp ears, W.M.: U sing problem generators to ex plore the effects of epistasis. In B¨ ack, T., ed.: Proceed in gs of the Seventh International Conference on Genetic Algorithms (ICGA97), San F rancisco, CA, Morgan K auf- mann (1997) 10 mk mk−cons mke mke−cons random 2e+04 5e+04 1e+05 2e+05 Policies comparison, MMDP problem type evaluations Fig. 4. Bo xplot (with logarithmic y axis) of the n umber of ev a lua tions needed to find the solution for M MDP . Three different v er sions of the multikult i mig ration po licy have b een tested here: mk , in which the mos t different individual is se nt; mk-cons , that sends the mo st different to the target’s consensus str ing , multi kulti- elite ( mke ) which chooses the mo st differe n t among the 5 0% most fit, and mk e- cons which sends the most different to the target’s cons ensus string a mong the 50% mos t fit. The r ightmost result cor resp onds to random mig ration. 11 2 4 6 8 10 12 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 Entropy Migration policy: multikulti−elite Cycles Entropy 0 5 10 15 20 0.5 1.0 1.5 2.0 2.5 3.0 Entropy Migration policy: send best Cycles Entropy 0 5 10 15 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Cycles Entropy mke b est a verag e Fig. 5. Entrop y (computed using the Shanno n formula H ( P ) = − P g ∈ P p ( f ( g )) l og b p ( f ( g )), where g is a mem b er of the p o pulation, f ( g ) its fitness, a nd p ( f ( g )) the frequency of that fitness a cross the whole p opula - tion) in a t ypical run of the MMDP problem, with the multikulti-elite migra tion po licy (left) and the b est migr ation p olicy (cen ter). E very line cor resp onds to a different p opulation, o f the eight r unning in parallel. The figure o n the right compare s a verage v alues, with the dashed line cor resp onding to the m ultikulti-elite exp eriment and the other to the exp er imen t that s e nds the b est individual.