Nemo: Multi-criteria test-suite minimization with integer nonlinear programming

Jun Wei Lin, Reyhaneh Jabbarvand, Joshua Garcia, Sam Malek

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Multi-criteria test-suite minimization aims to remove redundant test cases from a test suite based on some criteria such as code coverage, while trying to optimally maintain the capability of the reduced suite based on other criteria such as fault-detection effectiveness. Existing techniques addressing this problem with integer linear programming claim to produce optimal solutions. However, the multi-criteria test-suite minimization problem is inherently nonlinear, due to the fact that test cases are often dependent on each other in terms of test-case criteria. In this paper, we propose a framework that formulates the multi-criteria test-suite minimization problem as an integer nonlinear programming problem. To solve this problem optimally, we programmatically transform this nonlinear problem into a linear one and then solve the problem using modern linear solvers. We have implemented our framework as a tool, called Nemo, that supports a number of modern linear and nonlinear solvers. We have evaluated Nemo with a publicly available dataset and minimization problems involving multiple criteria including statement coverage, fault-revealing capability, and test execution time. The experimental results show that Nemo can be used to efficiently find an optimal solution for multi-criteria test-suite minimization problems with modern solvers, and the optimal solutions outperform the suboptimal ones by up to 164.29% in terms of the criteria considered in the problem.

Original languageEnglish (US)
Title of host publicationProceedings of the 40th International Conference on Software Engineering, ICSE 2018
PublisherIEEE Computer Society
Pages1039-1049
Number of pages11
ISBN (Electronic)9781450356381
DOIs
StatePublished - May 27 2018
Externally publishedYes
Event40th International Conference on Software Engineering, ICSE 2018 - Gothenburg, Sweden
Duration: May 27 2018Jun 3 2018

Publication series

NameProceedings - International Conference on Software Engineering
ISSN (Print)0270-5257

Conference

Conference40th International Conference on Software Engineering, ICSE 2018
Country/TerritorySweden
CityGothenburg
Period5/27/186/3/18

Keywords

  • Integer programming
  • Test-suite minimization

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Nemo: Multi-criteria test-suite minimization with integer nonlinear programming'. Together they form a unique fingerprint.

Cite this