An efficient format for nearly constant-time access to arbitrary time intervals in large trace files

Anthony Chan, William D Gropp, Ewing Lusk

Research output: Contribution to journalArticle

Abstract

A powerful method to aid in understanding the performance of parallel applications uses log or trace files containing time-stamped events and states (pairs of events). These trace files can be very large, often hundreds or even thousands of megabytes. Because of the cost of accessing and displaying such files, other methods are often used that reduce the size of the tracefiles at the cost of sacrificing detail or other information. This paper describes a hierarchical trace file format that provides for display of an arbitrary time window in a time independent of the total size of the file and roughly proportional to the number of events within the time window. This format eliminates the need to sacrifice data to achieve a smaller trace file size (since storage is inexpensive, it is necessary only to make efficient use of bandwidth to that storage). The format can be used to organize a trace file or to create a separate file of annotations that may be used with conventional trace files. We present an analysis of the time to access all of the events relevant to an interval of time and we describe experiments demonstrating the performance of this file format.

Original languageEnglish (US)
Pages (from-to)155-165
Number of pages11
JournalScientific Programming
Volume16
Issue number2-3
DOIs
StatePublished - 2008

Keywords

  • Message passing interface
  • Performance visualization
  • Trace file

ASJC Scopus subject areas

  • Software
  • Computer Science Applications

Fingerprint Dive into the research topics of 'An efficient format for nearly constant-time access to arbitrary time intervals in large trace files'. Together they form a unique fingerprint.

  • Cite this