TY - GEN
T1 - TestSage
T2 - 12th IEEE International Conference on Software Testing, Verification and Validation, ICST 2019
AU - Zhong, Hua
AU - Zhang, Lingming
AU - Khurshid, Sarfraz
N1 - Funding Information:
ACKNOWLEDGMENT We thank Dean Michael Berris, Eric Anderson and Ning Wangfor their advise and feedback on XRay, Moyang Zhang and Thanapong Lertpanyavit for their help on implementing TestSage.The project is partially supported by National Science Foundation grants CCF-1763906 and CCF-1718903.
Publisher Copyright:
© 2019 IEEE.
PY - 2019/4
Y1 - 2019/4
N2 - Regression testing is an important but expensive activity in software development. Among various types of tests, web service tests are usually one of the most expensive (due to network communications) but widely adopted types of tests in commercial software development. Regression test selection (RTS) aims to reduce the number of tests which need to be retested by only running tests that are affected by code changes. Although a large number of RTS techniques have been proposed in the past few decades, these techniques have not been adopted on large-scale web service testing. This is because most existing RTS techniques either require direct code dependency between tests and code under test or cannot be applied on large scale systems with enough efficiency. In this paper, we present a novel RTS technique, TestSage, that performs RTS for web service tests on large scale commercial software. With a small overhead, TestSage is able to collect fine grained (function level) dependency between test and service under test that do not directly depend on each other. TestSage has also been successfully applied to large complex systems with over a million functions. We conducted experiments of TestSage on a large scale backend service at Google. Experimental results show that TestSage reduces 34% of testing time when running all AEC (Analysis, Execution and Collection) phases, 50% of testing time while running without collection phase. TestSage has been integrated with internal testing framework at Google and runs day-to-day at the company.
AB - Regression testing is an important but expensive activity in software development. Among various types of tests, web service tests are usually one of the most expensive (due to network communications) but widely adopted types of tests in commercial software development. Regression test selection (RTS) aims to reduce the number of tests which need to be retested by only running tests that are affected by code changes. Although a large number of RTS techniques have been proposed in the past few decades, these techniques have not been adopted on large-scale web service testing. This is because most existing RTS techniques either require direct code dependency between tests and code under test or cannot be applied on large scale systems with enough efficiency. In this paper, we present a novel RTS technique, TestSage, that performs RTS for web service tests on large scale commercial software. With a small overhead, TestSage is able to collect fine grained (function level) dependency between test and service under test that do not directly depend on each other. TestSage has also been successfully applied to large complex systems with over a million functions. We conducted experiments of TestSage on a large scale backend service at Google. Experimental results show that TestSage reduces 34% of testing time when running all AEC (Analysis, Execution and Collection) phases, 50% of testing time while running without collection phase. TestSage has been integrated with internal testing framework at Google and runs day-to-day at the company.
KW - Regression Test Selection
KW - Regression Testing
KW - System Testing
KW - Web Service Testing
UR - http://www.scopus.com/inward/record.url?scp=85067966131&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85067966131&partnerID=8YFLogxK
U2 - 10.1109/ICST.2019.00052
DO - 10.1109/ICST.2019.00052
M3 - Conference contribution
AN - SCOPUS:85067966131
T3 - Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation, ICST 2019
SP - 430
EP - 440
BT - Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation, ICST 2019
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 22 April 2019 through 27 April 2019
ER -