Event log mining tool for large scale HPC systems

Ana Gainaru, Franck Cappello, Stefan Trausan-Matu, Bill Kramer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Event log files are the most common source of information for the characterization of events in large scale systems. However the large size of these files makes the task of manual analysing log messages to be difficult and error prone. This is the reason why recent research has been focusing on creating algorithms for automatically analysing these log files. In this paper we present a novel methodology for extracting templates that describe event formats from large datasets presenting an intuitive and user-friendly output to system administrators. Our algorithm is able to keep up with the rapidly changing environments by adapting the clusters to the incoming stream of events. For testing our tool, we have chosen 5 log files that have different formats and that challenge different aspects in the clustering task. The experiments show that our tool outperforms all other algorithms in all tested scenarios achieving an average precision and recall of 0.9, increasing the correct number of groups by a factor of 1.5 and decreasing the number of false positives and negatives by an average factor of 4.

Original languageEnglish (US)
Title of host publicationEuro-Par 2011 Parallel Processing - 17th International Conference, Proceedings
Pages52-64
Number of pages13
EditionPART 1
DOIs
StatePublished - Sep 8 2011
Event17th International Conference on Parallel Processing, Euro-Par 2011 - Bordeaux, France
Duration: Aug 29 2011Sep 2 2011

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume6852 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other17th International Conference on Parallel Processing, Euro-Par 2011
CountryFrance
CityBordeaux
Period8/29/119/2/11

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Event log mining tool for large scale HPC systems'. Together they form a unique fingerprint.

  • Cite this

    Gainaru, A., Cappello, F., Trausan-Matu, S., & Kramer, B. (2011). Event log mining tool for large scale HPC systems. In Euro-Par 2011 Parallel Processing - 17th International Conference, Proceedings (PART 1 ed., pp. 52-64). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6852 LNCS, No. PART 1). https://doi.org/10.1007/978-3-642-23400-2_6