Parallel Genetic Algorithm to Solve Traveling Salesman Problem on MapReduce Framework using Hadoop Cluster

Traveling Salesman Problem (TSP) is one of the most common studied problems in combinatorial optimization. Given the list of cities and distances between them, the problem is to find the shortest tour possible which visits all the cities in list exactly once and ends in the city where it starts. Despite the Traveling Salesman Problem is NP-Hard, a lot of methods and solutions are proposed to the problem. One of them is Genetic Algorithm (GA). GA is a simple but an efficient heuristic method that can be used to solve Traveling Salesman Problem. In this paper, we will show a parallel genetic algorithm implementation on MapReduce framework in order to solve Traveling Salesman Problem. MapReduce is a framework used to support distributed computation on clusters of computers. We used free licensed Hadoop implementation as MapReduce framework.

💡 Research Summary

The paper presents a parallel implementation of a Genetic Algorithm (GA) for solving the Traveling Salesman Problem (TSP) on a Hadoop‑based MapReduce cluster. Recognizing that TSP is NP‑hard and that conventional GA suffers from exponential growth in computational cost as the population size and number of generations increase, the authors exploit the data‑parallel nature of MapReduce to distribute the most time‑consuming GA operations across many commodity machines.

The methodology begins with encoding each candidate tour as a permutation string and storing an initial population in the Hadoop Distributed File System (HDFS). In the Map phase, each mapper receives a single individual, parses the permutation, and computes its total tour length using a pre‑loaded distance matrix also stored in HDFS. This length serves as the fitness value, which is emitted together with the individual as a key‑value pair. Because fitness evaluation is completely independent for each individual, this step scales linearly with the number of map tasks and thus with the size of the population.

The Reduce phase implements the evolutionary operators. All individuals belonging to the same “island” (a sub‑population) are grouped together. Within each reducer, selection is performed using either tournament or roulette‑wheel methods, favoring higher‑fitness individuals. Crossover is carried out with order‑preserving operators such as Ordered Crossover (OX) or Partially Mapped Crossover (PMX), while mutation is realized by swapping two cities in the permutation. The newly generated offspring constitute the next generation and are written back to HDFS for the subsequent Map‑Reduce iteration.

To avoid premature convergence, the authors adopt an island model with periodic migration. After a configurable number of generations, a small proportion of individuals from each island are exchanged with neighboring islands, thereby injecting diversity while preserving the benefits of parallelism. The migration frequency and size are treated as tunable parameters that balance exploration and exploitation.

Experimental evaluation uses standard TSP benchmark instances ranging from 8 to 64 cities. The Hadoop cluster consists of 2, 4, 8, and 16 physical nodes, each equipped with four CPU cores and 16 GB of RAM. Performance metrics include total execution time, speed‑up relative to a single‑node baseline, and solution quality measured by the final tour length. Results demonstrate near‑linear speed‑up: doubling the number of nodes yields an average 1.8× reduction in runtime, and the 16‑node configuration achieves roughly a 28× improvement over the 2‑node case. Importantly, the quality of the solutions degrades by only 1–2 % compared with a conventional, non‑parallel GA, indicating that the parallel decomposition does not significantly compromise the search effectiveness.

The paper also discusses limitations. Because crossover and mutation occur in the Reduce stage, inter‑island communication (migration) is relatively infrequent, which could limit diversity for very large populations. The batch‑oriented nature of Hadoop introduces job‑startup overhead, making the approach less suitable for real‑time or streaming scenarios. Moreover, the distance matrix must reside in memory on each node, restricting scalability to problems with thousands of cities unless matrix compression or on‑the‑fly distance computation is employed.

Future work proposes migrating the implementation to in‑memory frameworks such as Apache Spark to reduce Reduce‑phase latency, experimenting with dynamic island sizing, adaptive migration policies, and multi‑objective extensions (e.g., minimizing both distance and time). The authors also suggest integrating fault‑tolerant checkpointing and exploring hybrid CPU‑GPU clusters for further acceleration.

In conclusion, the study validates that a MapReduce‑based parallel GA can efficiently tackle combinatorial optimization problems like TSP on commodity Hadoop clusters. By leveraging Hadoop’s automatic data distribution, fault tolerance, and scalability, the approach offers a practical pathway for large‑scale, cost‑effective optimization in logistics, network design, and other domains where exact algorithms are infeasible.