Resurgence of Regression Test Selection for C++

Ben Fu, Sasa Misailovic, Milos Gligoric

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Regression testing - running available tests after each project change - is widely practiced in industry. Despite its widespread use and importance, regression testing is a costly activity. Regression test selection (RTS) optimizes regression testing by selecting only tests affected by project changes. RTS has been extensively studied and several tools have been deployed in large projects. However, work on RTS over the last decade has mostly focused on languages with abstract computing machines (e.g., JVM). Meanwhile development practices (e.g., frequency of commits, testing frameworks, compilers) in C++ projects have dramatically changed and the way we should design and implement RTS tools and the benefits of those tools is unknown. We present a design and implementation of an RTS technique, dubbed RTS++, that targets projects written in C++, which compile to LLVM IR and use the Google Test testing framework. RTS++ uses static analysis of a function call graph to select tests. RTS++ integrates with many existing build systems, including AutoMake, CMake, and Make. We evaluated RTS++ on 11 large open-source projects, totaling 3,811,916 lines of code. To the best of our knowledge, this is the largest evaluation of an RTS technique for C++. We measured the benefits of RTS++ compared to running all available tests (i.e., retest-all). Our results show that RTS++ reduces the number of executed tests and end-to-end testing time by 88% and 61% on average.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation, ICST 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages323-334
Number of pages12
ISBN (Electronic)9781728117355
DOIs
StatePublished - Apr 2019
Event12th IEEE International Conference on Software Testing, Verification and Validation, ICST 2019 - Xi'an, China
Duration: Apr 22 2019Apr 27 2019

Publication series

NameProceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation, ICST 2019

Conference

Conference12th IEEE International Conference on Software Testing, Verification and Validation, ICST 2019
CountryChina
CityXi'an
Period4/22/194/27/19

Keywords

  • Call graph
  • Google Test
  • LLVM
  • Regression test selection
  • Static analysis

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'Resurgence of Regression Test Selection for C++'. Together they form a unique fingerprint.

  • Cite this

    Fu, B., Misailovic, S., & Gligoric, M. (2019). Resurgence of Regression Test Selection for C++. In Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation, ICST 2019 (pp. 323-334). [8730161] (Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation, ICST 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICST.2019.00039