TY - GEN
T1 - Vectorizing Program Ingredients for Better JVM Testing
AU - Gao, Tianchang
AU - Chen, Junjie
AU - Zhao, Yingquan
AU - Zhang, Yuqun
AU - Zhang, Lingming
N1 - Publisher Copyright:
© 2023 ACM.
PY - 2023/7/12
Y1 - 2023/7/12
N2 - JVM testing is one of the most widely-used methodologies for guaranteeing the quality of JVMs. Among various JVM testing techniques, synthesis-based JVM testing, which constructs a test program by synthesizing various code snippets (also called program ingredients), has been demonstrated state-of-the-art. The existing synthesis-based JVM testing work puts more efforts in ensuring the validity of synthesized test programs, but ignores the influence of huge ingredient space, which largely limits the ingredient exploration efficiency as well as JVM testing performance. In this work, we propose Vectorized JVM Testing (called VECT) to further promote the performance of synthesis-based JVM testing. Its key insight is to reduce the huge ingredient space by clustering semantically similar ingredients via vectorizing ingredients using state-of-the-art code representation. To make VECT complete and more effective, based on vectorized ingredients, VECT further designs a feedback-driven ingredient selection strategy and an enhanced test oracle. We conducted an extensive study to evaluate VECT on three popular JVMs (i.e., HotSpot, OpenJ9, and Bisheng JDK) involving five OpenJDK versions. The results demonstrate VECT detects 115.03% ∼ 776.92% more unique inconsistencies than the state-of-the-art JVM testing technique during the same testing time. In particular, VECT detects 26 previously unknown bugs for them, 15 of which have already been confirmed/fixed by developers.
AB - JVM testing is one of the most widely-used methodologies for guaranteeing the quality of JVMs. Among various JVM testing techniques, synthesis-based JVM testing, which constructs a test program by synthesizing various code snippets (also called program ingredients), has been demonstrated state-of-the-art. The existing synthesis-based JVM testing work puts more efforts in ensuring the validity of synthesized test programs, but ignores the influence of huge ingredient space, which largely limits the ingredient exploration efficiency as well as JVM testing performance. In this work, we propose Vectorized JVM Testing (called VECT) to further promote the performance of synthesis-based JVM testing. Its key insight is to reduce the huge ingredient space by clustering semantically similar ingredients via vectorizing ingredients using state-of-the-art code representation. To make VECT complete and more effective, based on vectorized ingredients, VECT further designs a feedback-driven ingredient selection strategy and an enhanced test oracle. We conducted an extensive study to evaluate VECT on three popular JVMs (i.e., HotSpot, OpenJ9, and Bisheng JDK) involving five OpenJDK versions. The results demonstrate VECT detects 115.03% ∼ 776.92% more unique inconsistencies than the state-of-the-art JVM testing technique during the same testing time. In particular, VECT detects 26 previously unknown bugs for them, 15 of which have already been confirmed/fixed by developers.
KW - JVM Testing
KW - Java Virtual Machine
KW - Program Synthesis
KW - Test Oracle
UR - http://www.scopus.com/inward/record.url?scp=85167683938&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85167683938&partnerID=8YFLogxK
U2 - 10.1145/3597926.3598075
DO - 10.1145/3597926.3598075
M3 - Conference contribution
AN - SCOPUS:85167683938
T3 - ISSTA 2023 - Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis
SP - 526
EP - 537
BT - ISSTA 2023 - Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis
A2 - Just, Rene
A2 - Fraser, Gordon
PB - Association for Computing Machinery
T2 - 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2023
Y2 - 17 July 2023 through 21 July 2023
ER -