Approximate Transformations as Mutation Operators

Mutation testing is a well-established approach for evaluating test-suite quality by modifying code using syntax-changing (and potentially semantics-changing) transformations, called mutation operators. This paper proposes approximate transformations as new mutation operators that can give novel insights about the code and tests. Approximate transformations are semantics-changing transformations used in the emerging area of approximate computing, but so far they were not evaluated for mutation testing. We found that approximate transformations can be effective mutation operators. We compared three approximate transformations with a set of conventional mutation operators from the literature, on nine open-source Java subjects. The results showed that approximate transformations change program behavior differently from conventional mutation operators. Our analysis uncovered code patterns in which approximate mutants survivedand showed the practical value of approximate transformations for both understanding code amenable to approximations and discovering bad tests. We submitted 11 pull requests to fix bad tests. Seven have already been integrated by the developers.