TY - GEN
T1 - Mutation testing meets approximate computing
AU - Gligoric, Milos
AU - Khurshid, Sarfraz
AU - Misailovic, Sasa
AU - Shi, August
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/6/30
Y1 - 2017/6/30
N2 - One of the most widely studied techniques in software testing researchis mutation testing - a technique for evaluating the quality of testsuites. Despite over four decades of academic advances in thistechnique, mutation testing has not found its way to mainstreamdevelopment. The key issue with mutation testing is its highcomputational cost: It requires running the test suite against notjust the program under test but against typically thousands ofmutants, i.e., syntactic variants, of the program. Our key insight isthat exciting advances in the upcoming, yet unrelated, area ofapproximate computing allow us to define a principled approach thatprovides the benefits of traditional mutation testing at a fraction ofits usually large cost. This paper introduces the idea of a novel approach, named ApproxiMut, that blends the power of mutation testing with the practicality ofapproximate computing. To demonstrate the potential of our approach, we present a concrete instantiation: Rather than executing testsagainst each mutant on the exact program version, ApproxiMut obtainsan approximate test/program version by applying approximatetransformations and runs tests against each mutant on the approximatedversion. Our initial goal is to (1) measure the correlation betweenmutation scores on the exact and approximate program versions, (2)evaluate the relation among mutation operators and approximatetransformations, (3) discover the best way to approximate a test and aprogram, and (4) evaluate the benefits of ApproxiMut. Our preliminaryresults show similar mutation scores on the exact and approximateprogram versions and uncovered a case when an approximated test was, to our surprise, better than the exact test.
AB - One of the most widely studied techniques in software testing researchis mutation testing - a technique for evaluating the quality of testsuites. Despite over four decades of academic advances in thistechnique, mutation testing has not found its way to mainstreamdevelopment. The key issue with mutation testing is its highcomputational cost: It requires running the test suite against notjust the program under test but against typically thousands ofmutants, i.e., syntactic variants, of the program. Our key insight isthat exciting advances in the upcoming, yet unrelated, area ofapproximate computing allow us to define a principled approach thatprovides the benefits of traditional mutation testing at a fraction ofits usually large cost. This paper introduces the idea of a novel approach, named ApproxiMut, that blends the power of mutation testing with the practicality ofapproximate computing. To demonstrate the potential of our approach, we present a concrete instantiation: Rather than executing testsagainst each mutant on the exact program version, ApproxiMut obtainsan approximate test/program version by applying approximatetransformations and runs tests against each mutant on the approximatedversion. Our initial goal is to (1) measure the correlation betweenmutation scores on the exact and approximate program versions, (2)evaluate the relation among mutation operators and approximatetransformations, (3) discover the best way to approximate a test and aprogram, and (4) evaluate the benefits of ApproxiMut. Our preliminaryresults show similar mutation scores on the exact and approximateprogram versions and uncovered a case when an approximated test was, to our surprise, better than the exact test.
KW - approximate computing
KW - approximate transformations
KW - mutation testing
UR - http://www.scopus.com/inward/record.url?scp=85026744114&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85026744114&partnerID=8YFLogxK
U2 - 10.1109/ICSE-NIER.2017.15
DO - 10.1109/ICSE-NIER.2017.15
M3 - Conference contribution
AN - SCOPUS:85026744114
T3 - Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: New Ideas and Emerging Results Track, ICSE-NIER 2017
SP - 3
EP - 6
BT - Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 39th IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results Track, ICSE-NIER 2017
Y2 - 20 May 2017 through 28 May 2017
ER -