Scaling genetic algorithms using MapReduce

Abhishek Verma, Xavier Llorà, David E. Goldberg, Roy H. Campbell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Genetic algorithms(GAs) are increasingly being applied to large scale problems. The traditional MPI-based parallel GAs require detailed knowledge about machine architecture. On the other hand, MapReduce is a powerful abstraction proposed by Google for making scalable and fault tolerant applications. In this paper, we show how genetic algorithms can be modeled into the MapReduce model. We describe the algorithm design and implementation of GAs on Hadoop, an open source implementation of MapReduce. Our experiments demonstrate the convergence and scalability up to 105 variable problems. Adding more resources would enable us to solve even larger problems without any changes in the algorithms and implementation since we do not introduce any performance bottlenecks.

Original languageEnglish (US)
Title of host publicationISDA 2009 - 9th International Conference on Intelligent Systems Design and Applications
Pages13-18
Number of pages6
DOIs
StatePublished - Dec 1 2009
Event9th International Conference on Intelligent Systems Design and Applications, ISDA 2009 - Pisa, Italy
Duration: Nov 30 2009Dec 2 2009

Publication series

NameISDA 2009 - 9th International Conference on Intelligent Systems Design and Applications

Other

Other9th International Conference on Intelligent Systems Design and Applications, ISDA 2009
CountryItaly
CityPisa
Period11/30/0912/2/09

Keywords

  • Genetic algorithms
  • MapReduce
  • Scalability

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computational Theory and Mathematics
  • Signal Processing
  • Software

Fingerprint Dive into the research topics of 'Scaling genetic algorithms using MapReduce'. Together they form a unique fingerprint.

  • Cite this

    Verma, A., Llorà, X., Goldberg, D. E., & Campbell, R. H. (2009). Scaling genetic algorithms using MapReduce. In ISDA 2009 - 9th International Conference on Intelligent Systems Design and Applications (pp. 13-18). [5362925] (ISDA 2009 - 9th International Conference on Intelligent Systems Design and Applications). https://doi.org/10.1109/ISDA.2009.181