Advances in Benchmarking Techniques: New Standards and Quantitative Metrics

Thomas M. Conte, Wen Mei W. Hwu

Research output: Contribution to journalArticle

Abstract

Comparing computer system performance is difficult. Techniques for benchmarking computers have evolved from small test programs into extensive specifications of workloads. Over the last several years, competing vendors in several major computer markets have banded together to form consortia to develop standard benchmark sets. In addition to system evaluation, benchmarking is increasingly being used for system design. Characterizing the benchmarks in terms of system design requirements aids both of these activities. This chapter discusses new advances in benchmarking, including quantitative benchmark characterization. An overview of several popular, contemporary benchmark suites is presented, along with a discussion of the different philosophies of each suite. A detailed characterization is presented for a popular workstation benchmark suite—SPECint92. These characteristics focus on memory system and processor microarchitecture design, since these two subsystems largely determine the performance for workstation class machines. Although comprehensive in its hardware considerations, software interactions are not covered by these characteristics. Suggestions for software characteristics are discussed at the close of the chapter.

Original languageEnglish (US)
Pages (from-to)231-253
Number of pages23
JournalAdvances in Computers
Volume41
Issue numberC
DOIs
StatePublished - Jan 1 1995

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Advances in Benchmarking Techniques: New Standards and Quantitative Metrics'. Together they form a unique fingerprint.

  • Cite this