An Empirical Comparison of Mutant Selection Assessment Metrics

Jie M. Zhang, Lingming Zhang, Dan Hao, Lu Zhang, Mark Harman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Mutation testing is expensive due to the large number of mutants, a problem typically tackled using selective techniques, thereby raising the fundamental question of how to evaluate the selection process. Existing mutant selection approaches rely on one of two types of metrics (or assessment criteria), one based on adequate test sets and the other based on inadequate test sets. This raises the question as to whether these two metrics are correlated, complementary or substitutable for one another. The tester's faith in mutant selection as well as the validity of previous research work using only one metric rely on the answer to this question, yet it currently remains unanswered. To answer it, we perform qualitative and quantitative comparisons with 104 different projects, consisting of over 600,000 lines of code. Our results indicate a strong connection between the two types of metrics (R2=0.8622 on average). The strategy for dealing with equivalent mutants and test density is observed to have a negligible impact for mutant selection.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages90-101
Number of pages12
ISBN (Electronic)9781728108889
DOIs
StatePublished - Apr 2019
Externally publishedYes
Event12th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019 - Xi'an, China
Duration: Apr 22 2019Apr 27 2019

Publication series

NameProceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019

Conference

Conference12th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019
Country/TerritoryChina
CityXi'an
Period4/22/194/27/19

Keywords

  • Assessment metrics
  • Mutant selection
  • Mutation testing

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality

Fingerprint

Dive into the research topics of 'An Empirical Comparison of Mutant Selection Assessment Metrics'. Together they form a unique fingerprint.

Cite this