Metadata traces and workload models for evaluating big storage systems

Cristina L. Abad, Huong Luu, Nathan Roberts, Kihwal Lee, Yi Lu, Roy H. Campbell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Efficient namespace metadata management is increasingly important as next-generation file systems are designed for peta and exascales. New schemes have been proposed; however, their evaluation has been insufficient due to a lack of appropriate namespace metadata traces. Specifically, no Big Data storage system metadata trace is publicly available and existing ones are a poor replacement. We studied publicly available traces and one Big Data trace from Yahoo! and note some of the differences and their implications to metadata management studies. We discuss the insufficiency of existing evaluation approaches and present a first step towards a statistical metadata workload model that can capture the relevant characteristics of a workload and is suitable for synthetic workload generation. We describe Mimesis, a synthetic workload generator, and evaluate its usefulness through a case study in a least recently used metadata cache for the Hadoop Distributed File System. Simulation results show that the traces generated by Mimesis mimic the original workload and can be used in place of the real trace providing accurate results.

Original languageEnglish (US)
Title of host publicationProceedings - 2012 IEEE/ACM 5th International Conference on Utility and Cloud Computing, UCC 2012
Pages125-132
Number of pages8
DOIs
StatePublished - Dec 1 2012
Event2012 IEEE/ACM 5th International Conference on Utility and Cloud Computing, UCC 2012 - Chicago, IL, United States
Duration: Nov 5 2012Nov 8 2012

Publication series

NameProceedings - 2012 IEEE/ACM 5th International Conference on Utility and Cloud Computing, UCC 2012

Other

Other2012 IEEE/ACM 5th International Conference on Utility and Cloud Computing, UCC 2012
CountryUnited States
CityChicago, IL
Period11/5/1211/8/12

    Fingerprint

Keywords

  • Big data
  • HDFS
  • MDS
  • Metadata
  • Storage

ASJC Scopus subject areas

  • Software

Cite this

Abad, C. L., Luu, H., Roberts, N., Lee, K., Lu, Y., & Campbell, R. H. (2012). Metadata traces and workload models for evaluating big storage systems. In Proceedings - 2012 IEEE/ACM 5th International Conference on Utility and Cloud Computing, UCC 2012 (pp. 125-132). [6424937] (Proceedings - 2012 IEEE/ACM 5th International Conference on Utility and Cloud Computing, UCC 2012). https://doi.org/10.1109/UCC.2012.27