History-Driven Test Program Synthesis for JVM Testing

Yingquan Zhao, Zan Wang, Junjie Chen, Mengdi Liu, Mingyuan Wu, Yuqun Zhang, Lingming Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Java Virtual Machine (JVM) provides the runtime environment for Java programs, which allows Java to be 'write once, run anywhere'. JVM plays a decisive role in the correctness of all Java programs running on it. Therefore, ensuring the correctness and robustness of JVM implementations is essential for Java programs. To date, various techniques have been proposed to expose JVM bugs via generating potential bug-revealing test programs. However, the diversity and effectiveness of test programs generated by existing research are far from enough since they mainly focus on minor syntactic/semantic mutations. In this paper, we propose JavaTailor, the first history-driven test program synthesis technique, which synthesizes diverse test programs by weaving the ingredients extracted from JVM historical bug-revealing test programs into seed programs for covering more JVM behaviors/paths. More specifically, JavaTailor first extracts five types of code ingredients from the historical bug-revealing test programs. Then, to synthesize diverse test programs, it iteratively inserts the extracted ingredients into the seed programs and strengthens their interactions via introducing extra data dependencies between them. Finally, JavaTailor employs these synthesized test programs to differentially test JVMs. Our experimental results on popular JVM implementations (i.e., HotSpot and OpenJ9) show that JavaTailor outperforms the state-of-the-art technique in generating more diverse and effective test programs, e.g., test programs generated by JavaTailor can achieve higher JVM code coverage and detect many more unique inconsistencies than the state-of-the-art technique. Furthermore, JavaTailor has detected 10 previously unknown bugs, 6 of which have been confirmed/fixed by developers.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 ACM/IEEE 44th International Conference on Software Engineering, ICSE 2022
PublisherIEEE Computer Society
Pages1133-1144
Number of pages12
ISBN (Electronic)9781450392211
DOIs
StatePublished - 2022
Event44th ACM/IEEE International Conference on Software Engineering, ICSE 2022 - Pittsburgh, United States
Duration: May 22 2022May 27 2022

Publication series

NameProceedings - International Conference on Software Engineering
Volume2022-May
ISSN (Print)0270-5257

Conference

Conference44th ACM/IEEE International Conference on Software Engineering, ICSE 2022
Country/TerritoryUnited States
CityPittsburgh
Period5/22/225/27/22

Keywords

  • Compiler Testing
  • JVM Testing
  • Java Virtual Machine
  • Program Synthesis

ASJC Scopus subject areas

  • Software

Cite this