We present a novel distributed evolutionary algorithm, KaFFPaE, to solve the Graph Partitioning Problem, which makes use of KaFFPa (Karlsruhe Fast Flow Partitioner). The use of our multilevel graph partitioner KaFFPa provides new effective crossover and mutation operators. By combining these with a scalable communication protocol we obtain a system that is able to improve the best known partitioning results for many inputs in a very short amount of time. For example, in Walshaw's well known benchmark tables we are able to improve or recompute 76% of entries for the tables with 1%, 3% and 5% imbalance.
Problems of graph partitioning arise in various areas of computer science, engineering, and related fields. For example in high performance computing [27], community detection in social networks [25] and route planning [4]. In particular the graph partitioning problem is very valuable for parallel computing. In this area, graph partitioning is mostly used to partition the underlying graph model of computation and communication. Roughly speaking, vertices in this graph represent computation units and edges denote communication. This graph needs to be partitioned such that there are few edges between the blocks (pieces). In particular, if we want to use k processors we want to partition the graph into k blocks of about equal size.
In this paper we focus on a version of the problem that constrains the maximum block size to (1 + ) times the average block size and tries to minimize the total cut size, i.e., the number of edges that run between blocks. It is well known that this problem is NP-complete [7] and that there is no approximation algorithm with a constant ratio factor for general graphs [7]. Therefore mostly heuristic algorithms are used in practice.
A successful heuristic for partitioning large graphs is the multilevel graph partitioning (MGP) approach depicted in Figure 1 where the graph is recursively contracted to achieve smaller graphs which should reflect the same basic structure as the input graph. After applying an initial partitioning algorithm to the smallest graph, the contraction is undone and, at each level, a local refinement method is used to improve the partitioning induced by the coarser level.
The main focus of this paper is a technique which integrates an evolutionary search algorithm with our multilevel graph partitioner KaFFPa and its scalable parallelization. We present novel mutation and combine operators which in contrast to previous methods that use a graph partitioner [28,11] do not need random perturbations of edge weights. We show in Section 6 that the usage of edge weight perturbations decreases the overall quality of the underlying graph partitioner. The new combine operators enable us to combine individuals of different kinds (see Section 4 for more details). Due to the parallelization our system is able to compute partitions that have quality comparable or better than previous entries in Walshaw’s well known partitioning benchmark within a few minutes for graphs of moderate size. Previous methods of Soper et.al [28] required runtimes of up to one week for graphs of that size. We therefore believe that in contrast to previous methods, our method is very valuable in the area of high performance computing.
The paper is organized as follows. We begin in Section 2 by introducing basic concepts. After shortly presenting Related Work in Section 3, we continue describing the main evolutionary components in Section 4 and its parallelization in Section 5. A summary of extensive experiments done to tune the algorithm and evaluate its performance is presented in Section 6. A brief outline of the techniques used in the multilevel graph partitioner KaFFPa is provided in Appendix A. We have implemented these techniques in the graph partitioner KaFFPaE (Karlsruhe Fast Flow Partitioner Evolutionary) which is written in C++. Experiments reported in Section 6 indicate that KaFFPaE is able to compute partitions of very high quality and scales well to large networks and machines.
Consider an undirected graph G = (V, E, c, ω) with edge weights ω : E → R >0 , node weights c : V → R ≥0 , n = |V |, and m = |E|. We extend c and ω to sets, i.e., c(V ) := v∈V c(v) and ω(E ) := e∈E ω(e). Γ(v) := {u : {v, u} ∈ E} denotes the neighbors of v. We are looking for blocks of nodes V 1 ,. . . ,V k that partition
The balancing constraint demands that ∀i ∈ {1..k} : c(V i ) ≤ L max := (1 + )c(V )/k + max v∈V c(v) for some parameter . The last term in this equation arises because each node is atomic and therefore a deviation of the heaviest node has to be allowed. The objective is to minimize the total cut i<j w(E ij ) where E ij := {{u, v} ∈ E : u ∈ V i , v ∈ V j }. A clustering is also a partition of the nodes, however k is usually not given in advance and the balance constraint is removed. A vertex v ∈ V i that has a neighbor w ∈ V j , i = j, is a boundary vertex. An abstract view of the partitioned graph is the so called quotient graph, where vertices represent blocks and edges are induced by connectivity between blocks. Given two clusterings C 1 and C 2 the overlay clustering is the clustering where each block corresponds to a connected component of the graph G E = (V, E\E) where E is the union of the cut edges of C 1 and C 2 , i.e. all edges that run between blocks in either C 1 or C 2 . By default, our initial inputs will have unit edge and node weights. However, even those will be translated into weighted problems in the course of the algorithm.
A matching M ⊆ E is a set of edges that do not share
This content is AI-processed based on open access ArXiv data.
