TY - GEN
T1 - Isolating failure-inducing combinations in combinatorial testing using test augmentation and classification
AU - Shakya, Kiran
AU - Xie, Tao
AU - Li, Nuo
AU - Lei, Yu
AU - Kacker, Raghu
AU - Kuhn, Richard
PY - 2012
Y1 - 2012
N2 - Combinatorial Testing (CT) is a systematic way of sampling input parameters of the software under test (SUT). A t-way combinatorial test set can exercise all behaviors of the SUT caused by interactions between t input parameters or less. Although combinatorial testing can provide fault detection capability, it is often desirable to isolate the input combinations that cause failures. Isolating these failure-inducing combinations aids developers in understanding the causes of failures. Previous work directly uses classification tree analysis on the results of combinatorial testing to model the failure inducing combinations. But in many scenarios, the effectiveness of classification depends upon whether the analyzed test set is sufficient for classification. In addition, generating combinatorial tests for more-than-6-way combination is generally expensive. To address these issues, we propose an approach that uses existing combinatorial testing results to generate additional tests that enhance the effectiveness of classification. In addition, our approach also includes a technique to reduce the complexity of the resulting classification tree so that developers can understand the nature of failure-inducing combinations. We present the preliminary results of our approach applied on the TCAS benchmark.
AB - Combinatorial Testing (CT) is a systematic way of sampling input parameters of the software under test (SUT). A t-way combinatorial test set can exercise all behaviors of the SUT caused by interactions between t input parameters or less. Although combinatorial testing can provide fault detection capability, it is often desirable to isolate the input combinations that cause failures. Isolating these failure-inducing combinations aids developers in understanding the causes of failures. Previous work directly uses classification tree analysis on the results of combinatorial testing to model the failure inducing combinations. But in many scenarios, the effectiveness of classification depends upon whether the analyzed test set is sufficient for classification. In addition, generating combinatorial tests for more-than-6-way combination is generally expensive. To address these issues, we propose an approach that uses existing combinatorial testing results to generate additional tests that enhance the effectiveness of classification. In addition, our approach also includes a technique to reduce the complexity of the resulting classification tree so that developers can understand the nature of failure-inducing combinations. We present the preliminary results of our approach applied on the TCAS benchmark.
KW - Classification Tree
KW - Combinatorial Testing
KW - Fault Localization
UR - http://www.scopus.com/inward/record.url?scp=84862309798&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84862309798&partnerID=8YFLogxK
U2 - 10.1109/ICST.2012.149
DO - 10.1109/ICST.2012.149
M3 - Conference contribution
AN - SCOPUS:84862309798
SN - 9780769546704
T3 - Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012
SP - 620
EP - 623
BT - Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012
T2 - 5th IEEE International Conference on Software Testing, Verification and Validation, ICST 2012
Y2 - 17 April 2012 through 21 April 2012
ER -