Combinatorial generation of structurally complex test inputs for commercial software applications

Hua Zhong, Lingming Zhang, Sarfraz Khurshid

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Despite recent progress in automated test generation research, significant challenges remain for applying these techniques on large-scale software systems. Large-scale software systems under test often require structurally complex test inputs within a large input domain. It is challenging to automatically generate a reasonable number of tests that are both legal and behaviorally-diverse to exercise these systems. Constraint-based test generation is an effective approach for generating structurally complex inputs for systematic testing. While this approach can typically generate large numbers of tests, it has limited scalability-tests generated by this approach are usually only up to a small bound on input size. Combinatorial test generation, e.g., pair-wise testing, is a more scalable approach but is challenging to apply on commercial software systems that require complex input structures that cannot be formed by using arbitrary combinations. This paper introduces comKorat, which unifies constraint-based generation of structurally complex tests with combinatorial test generation methods. Specifi-cally, comKorat integrates Korat and ACTS test generators to generate test suites for large-scale software systems with structurally complex test inputs. We have successfully applied comKorat on four large-scale software applications developed at eBay and Yahoo!. The experimental results show that comKorat outperforms existing solutions in execution time and test coverage. Furthermore, comKorat found a total of 59 previously unknown bugs in the above four applications.

Original languageEnglish (US)
Title of host publicationFSE 2016 - Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering
EditorsZhendong Su, Thomas Zimmermann, Jane Cleland-Huang
PublisherAssociation for Computing Machinery
Pages981-986
Number of pages6
ISBN (Electronic)9781450342186
DOIs
StatePublished - Nov 1 2016
Externally publishedYes
Event24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016 - Seattle, United States
Duration: Nov 13 2016Nov 18 2016

Publication series

NameProceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering
Volume13-18-November-2016

Other

Other24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016
Country/TerritoryUnited States
CitySeattle
Period11/13/1611/18/16

Keywords

  • ACTS
  • Combinatorial test generation
  • Constraint-based test generation
  • Korat

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Combinatorial generation of structurally complex test inputs for commercial software applications'. Together they form a unique fingerprint.

Cite this