Code coverage and input variability: Effects on architecture and compiler research

Hillery C. Hunter, Wen-Mei W Hwu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Meaningful application benchmarking is crucial to processor design space exploration and compiler development. Many recent studies in the embedded processor domain have used reference telecommunications C programs which were originally intended to verify product compliance with a standard. In this paper, we demonstrate that telecommunications reference applications include a significant amount of superfluous code that skews a wide variety of experiments related to code size. A categorization of extra code is presented, and it is also demonstrated that benchmark inputs do not test broad usage patterns that might exist in a real system. It is shown that care must be taken in future studies to properly construct benchmarks which match the intended purpose of the target system and to provide inputs that more thoroughly exercise paths through telecommunications applications.

Original languageEnglish (US)
Title of host publicationProceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02
Pages79-87
Number of pages9
DOIs
StatePublished - Dec 1 2002
Event2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02 - Grenoble, France
Duration: Oct 8 2002Oct 11 2002

Publication series

NameProceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02

Other

Other2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02
CountryFrance
CityGrenoble
Period10/8/0210/11/02

Fingerprint

Telecommunication
Benchmarking
Experiments
Compliance

Keywords

  • Architecture
  • Benchmarks
  • Code coverage
  • Compiler
  • DSP
  • Telecommunications

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering

Cite this

Hunter, H. C., & Hwu, W-M. W. (2002). Code coverage and input variability: Effects on architecture and compiler research. In Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02 (pp. 79-87). (Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02). https://doi.org/10.1145/581630.581643

Code coverage and input variability : Effects on architecture and compiler research. / Hunter, Hillery C.; Hwu, Wen-Mei W.

Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02. 2002. p. 79-87 (Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hunter, HC & Hwu, W-MW 2002, Code coverage and input variability: Effects on architecture and compiler research. in Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02. Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02, pp. 79-87, 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02, Grenoble, France, 10/8/02. https://doi.org/10.1145/581630.581643
Hunter HC, Hwu W-MW. Code coverage and input variability: Effects on architecture and compiler research. In Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02. 2002. p. 79-87. (Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02). https://doi.org/10.1145/581630.581643
Hunter, Hillery C. ; Hwu, Wen-Mei W. / Code coverage and input variability : Effects on architecture and compiler research. Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02. 2002. pp. 79-87 (Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02).
@inproceedings{d02809479b5f4f9fb49b121e12905d15,
title = "Code coverage and input variability: Effects on architecture and compiler research",
abstract = "Meaningful application benchmarking is crucial to processor design space exploration and compiler development. Many recent studies in the embedded processor domain have used reference telecommunications C programs which were originally intended to verify product compliance with a standard. In this paper, we demonstrate that telecommunications reference applications include a significant amount of superfluous code that skews a wide variety of experiments related to code size. A categorization of extra code is presented, and it is also demonstrated that benchmark inputs do not test broad usage patterns that might exist in a real system. It is shown that care must be taken in future studies to properly construct benchmarks which match the intended purpose of the target system and to provide inputs that more thoroughly exercise paths through telecommunications applications.",
keywords = "Architecture, Benchmarks, Code coverage, Compiler, DSP, Telecommunications",
author = "Hunter, {Hillery C.} and Hwu, {Wen-Mei W}",
year = "2002",
month = "12",
day = "1",
doi = "10.1145/581630.581643",
language = "English (US)",
isbn = "1581135750",
series = "Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02",
pages = "79--87",
booktitle = "Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02",

}

TY - GEN

T1 - Code coverage and input variability

T2 - Effects on architecture and compiler research

AU - Hunter, Hillery C.

AU - Hwu, Wen-Mei W

PY - 2002/12/1

Y1 - 2002/12/1

N2 - Meaningful application benchmarking is crucial to processor design space exploration and compiler development. Many recent studies in the embedded processor domain have used reference telecommunications C programs which were originally intended to verify product compliance with a standard. In this paper, we demonstrate that telecommunications reference applications include a significant amount of superfluous code that skews a wide variety of experiments related to code size. A categorization of extra code is presented, and it is also demonstrated that benchmark inputs do not test broad usage patterns that might exist in a real system. It is shown that care must be taken in future studies to properly construct benchmarks which match the intended purpose of the target system and to provide inputs that more thoroughly exercise paths through telecommunications applications.

AB - Meaningful application benchmarking is crucial to processor design space exploration and compiler development. Many recent studies in the embedded processor domain have used reference telecommunications C programs which were originally intended to verify product compliance with a standard. In this paper, we demonstrate that telecommunications reference applications include a significant amount of superfluous code that skews a wide variety of experiments related to code size. A categorization of extra code is presented, and it is also demonstrated that benchmark inputs do not test broad usage patterns that might exist in a real system. It is shown that care must be taken in future studies to properly construct benchmarks which match the intended purpose of the target system and to provide inputs that more thoroughly exercise paths through telecommunications applications.

KW - Architecture

KW - Benchmarks

KW - Code coverage

KW - Compiler

KW - DSP

KW - Telecommunications

UR - http://www.scopus.com/inward/record.url?scp=77952985184&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77952985184&partnerID=8YFLogxK

U2 - 10.1145/581630.581643

DO - 10.1145/581630.581643

M3 - Conference contribution

AN - SCOPUS:77952985184

SN - 1581135750

SN - 9781581135756

T3 - Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02

SP - 79

EP - 87

BT - Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, CASES '02

ER -