Approximate Transformations as Mutation Operators

Farah Hariri, August Shi, Owolabi Legunsen, Milos Gligoric, Sarfraz Khurshid, Sasa Misailovic

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Mutation testing is a well-established approach for evaluating test-suite quality by modifying code using syntax-changing (and potentially semantics-changing) transformations, called mutation operators. This paper proposes approximate transformations as new mutation operators that can give novel insights about the code and tests. Approximate transformations are semantics-changing transformations used in the emerging area of approximate computing, but so far they were not evaluated for mutation testing. We found that approximate transformations can be effective mutation operators. We compared three approximate transformations with a set of conventional mutation operators from the literature, on nine open-source Java subjects. The results showed that approximate transformations change program behavior differently from conventional mutation operators. Our analysis uncovered code patterns in which approximate mutants survivedand showed the practical value of approximate transformations for both understanding code amenable to approximations and discovering bad tests. We submitted 11 pull requests to fix bad tests. Seven have already been integrated by the developers.

Original languageEnglish (US)
Title of host publicationProceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages285-296
Number of pages12
ISBN (Electronic)9781538650127
DOIs
StatePublished - May 25 2018
Event11th IEEE International Conference on Software Testing, Verification and Validation, ICST 2018 - Vasteras, Sweden
Duration: Apr 9 2018Apr 13 2018

Publication series

NameProceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018

Other

Other11th IEEE International Conference on Software Testing, Verification and Validation, ICST 2018
CountrySweden
CityVasteras
Period4/9/184/13/18

Fingerprint

Semantics
Testing

Keywords

  • approximate computing
  • mutation testing
  • software testing

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality

Cite this

Hariri, F., Shi, A., Legunsen, O., Gligoric, M., Khurshid, S., & Misailovic, S. (2018). Approximate Transformations as Mutation Operators. In Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018 (pp. 285-296). (Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICST.2018.00036

Approximate Transformations as Mutation Operators. / Hariri, Farah; Shi, August; Legunsen, Owolabi; Gligoric, Milos; Khurshid, Sarfraz; Misailovic, Sasa.

Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 285-296 (Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hariri, F, Shi, A, Legunsen, O, Gligoric, M, Khurshid, S & Misailovic, S 2018, Approximate Transformations as Mutation Operators. in Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018. Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018, Institute of Electrical and Electronics Engineers Inc., pp. 285-296, 11th IEEE International Conference on Software Testing, Verification and Validation, ICST 2018, Vasteras, Sweden, 4/9/18. https://doi.org/10.1109/ICST.2018.00036
Hariri F, Shi A, Legunsen O, Gligoric M, Khurshid S, Misailovic S. Approximate Transformations as Mutation Operators. In Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 285-296. (Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018). https://doi.org/10.1109/ICST.2018.00036
Hariri, Farah ; Shi, August ; Legunsen, Owolabi ; Gligoric, Milos ; Khurshid, Sarfraz ; Misailovic, Sasa. / Approximate Transformations as Mutation Operators. Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 285-296 (Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018).
@inproceedings{ce0ae1c90b1649f39bd14f81f7a07b31,
title = "Approximate Transformations as Mutation Operators",
abstract = "Mutation testing is a well-established approach for evaluating test-suite quality by modifying code using syntax-changing (and potentially semantics-changing) transformations, called mutation operators. This paper proposes approximate transformations as new mutation operators that can give novel insights about the code and tests. Approximate transformations are semantics-changing transformations used in the emerging area of approximate computing, but so far they were not evaluated for mutation testing. We found that approximate transformations can be effective mutation operators. We compared three approximate transformations with a set of conventional mutation operators from the literature, on nine open-source Java subjects. The results showed that approximate transformations change program behavior differently from conventional mutation operators. Our analysis uncovered code patterns in which approximate mutants survivedand showed the practical value of approximate transformations for both understanding code amenable to approximations and discovering bad tests. We submitted 11 pull requests to fix bad tests. Seven have already been integrated by the developers.",
keywords = "approximate computing, mutation testing, software testing",
author = "Farah Hariri and August Shi and Owolabi Legunsen and Milos Gligoric and Sarfraz Khurshid and Sasa Misailovic",
year = "2018",
month = "5",
day = "25",
doi = "10.1109/ICST.2018.00036",
language = "English (US)",
series = "Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "285--296",
booktitle = "Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018",
address = "United States",

}

TY - GEN

T1 - Approximate Transformations as Mutation Operators

AU - Hariri, Farah

AU - Shi, August

AU - Legunsen, Owolabi

AU - Gligoric, Milos

AU - Khurshid, Sarfraz

AU - Misailovic, Sasa

PY - 2018/5/25

Y1 - 2018/5/25

N2 - Mutation testing is a well-established approach for evaluating test-suite quality by modifying code using syntax-changing (and potentially semantics-changing) transformations, called mutation operators. This paper proposes approximate transformations as new mutation operators that can give novel insights about the code and tests. Approximate transformations are semantics-changing transformations used in the emerging area of approximate computing, but so far they were not evaluated for mutation testing. We found that approximate transformations can be effective mutation operators. We compared three approximate transformations with a set of conventional mutation operators from the literature, on nine open-source Java subjects. The results showed that approximate transformations change program behavior differently from conventional mutation operators. Our analysis uncovered code patterns in which approximate mutants survivedand showed the practical value of approximate transformations for both understanding code amenable to approximations and discovering bad tests. We submitted 11 pull requests to fix bad tests. Seven have already been integrated by the developers.

AB - Mutation testing is a well-established approach for evaluating test-suite quality by modifying code using syntax-changing (and potentially semantics-changing) transformations, called mutation operators. This paper proposes approximate transformations as new mutation operators that can give novel insights about the code and tests. Approximate transformations are semantics-changing transformations used in the emerging area of approximate computing, but so far they were not evaluated for mutation testing. We found that approximate transformations can be effective mutation operators. We compared three approximate transformations with a set of conventional mutation operators from the literature, on nine open-source Java subjects. The results showed that approximate transformations change program behavior differently from conventional mutation operators. Our analysis uncovered code patterns in which approximate mutants survivedand showed the practical value of approximate transformations for both understanding code amenable to approximations and discovering bad tests. We submitted 11 pull requests to fix bad tests. Seven have already been integrated by the developers.

KW - approximate computing

KW - mutation testing

KW - software testing

UR - http://www.scopus.com/inward/record.url?scp=85048385458&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048385458&partnerID=8YFLogxK

U2 - 10.1109/ICST.2018.00036

DO - 10.1109/ICST.2018.00036

M3 - Conference contribution

AN - SCOPUS:85048385458

T3 - Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018

SP - 285

EP - 296

BT - Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -