Mutation testing meets approximate computing

Milos Gligoric, Sarfraz Khurshid, Sasa Misailovic, August Shi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

One of the most widely studied techniques in software testing researchis mutation testing - a technique for evaluating the quality of testsuites. Despite over four decades of academic advances in thistechnique, mutation testing has not found its way to mainstreamdevelopment. The key issue with mutation testing is its highcomputational cost: It requires running the test suite against notjust the program under test but against typically thousands ofmutants, i.e., syntactic variants, of the program. Our key insight isthat exciting advances in the upcoming, yet unrelated, area ofapproximate computing allow us to define a principled approach thatprovides the benefits of traditional mutation testing at a fraction ofits usually large cost. This paper introduces the idea of a novel approach, named ApproxiMut, that blends the power of mutation testing with the practicality ofapproximate computing. To demonstrate the potential of our approach, we present a concrete instantiation: Rather than executing testsagainst each mutant on the exact program version, ApproxiMut obtainsan approximate test/program version by applying approximatetransformations and runs tests against each mutant on the approximatedversion. Our initial goal is to (1) measure the correlation betweenmutation scores on the exact and approximate program versions, (2)evaluate the relation among mutation operators and approximatetransformations, (3) discover the best way to approximate a test and aprogram, and (4) evaluate the benefits of ApproxiMut. Our preliminaryresults show similar mutation scores on the exact and approximateprogram versions and uncovered a case when an approximated test was, to our surprise, better than the exact test.

Original languageEnglish (US)
Title of host publicationProceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering
Subtitle of host publicationNew Ideas and Emerging Results Track, ICSE-NIER 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3-6
Number of pages4
ISBN (Electronic)9781538626757
DOIs
StatePublished - Jun 30 2017
Event39th IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results Track, ICSE-NIER 2017 - Buenos Aires, Argentina
Duration: May 20 2017May 28 2017

Publication series

NameProceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: New Ideas and Emerging Results Track, ICSE-NIER 2017

Other

Other39th IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results Track, ICSE-NIER 2017
Country/TerritoryArgentina
CityBuenos Aires
Period5/20/175/28/17

Keywords

  • approximate computing
  • approximate transformations
  • mutation testing

ASJC Scopus subject areas

  • Organizational Behavior and Human Resource Management
  • Software
  • Management of Technology and Innovation

Fingerprint

Dive into the research topics of 'Mutation testing meets approximate computing'. Together they form a unique fingerprint.

Cite this