History-Driven Test Program Synthesis for JVM Testing

Yingquan Zhao1; Zan Wang1; Junjie Chen1; Mengdi Liu1; Mingyuan Wu2; Yuqun Zhang2; Lingming Zhang3

10.1145/3510003.3510059

2022

ACM/IEEE 44th International Conference on Software Engineering (ICSE)

0270-5257

978-1-6654-9589-9

2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE)

2022-May

1133-1144

25-27 May 2022

Pittsburgh, PA, USA

10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA

IEEE COMPUTER SOC

Java Virtual Machine (JVM) provides the runtime environment for Java programs, which allows Java to be "write once, run anywhere". JVM plays a decisive role in the correctness of all Java programs running on it. Therefore, ensuring the correctness and robustness of JVM implementations is essential for Java programs. To date, various techniques have been proposed to expose JVM bugs via generating potential bug-revealing test programs. However, the diversity and effectiveness of test programs generated by existing research are far from enough since they mainly focus on minor syntactic/semantic mutations. In this paper, we propose JavaTailor, the first history-driven test program synthesis technique, which synthesizes diverse test programs by weaving the ingredients extracted from JVM historical bug-revealing test programs into seed programs for covering more JVM behaviors/paths. More specifically, JavaTailor first extracts five types of code ingredients from the historical bug-revealing test programs. Then, to synthesize diverse test programs, it iteratively inserts the extracted ingredients into the seed programs and strengthens their interactions via introducing extra data dependencies between them. Finally, JavaTailor employs these synthesized test programs to differentially test JVMs. Our experimental results on popular JVM implementations (i.e., HotSpot and OpenJ9) show that JavaTailor outperforms the state-of-the-art technique in generating more diverse and effective test programs, e.g., test programs generated by JavaTailor can achieve higher JVM code coverage and detect many more unique inconsistencies than the state-of-the-art technique. Furthermore, JavaTailor has detected 10 previously unknown bugs, 6 of which have been confirmed/fixed by developers.

Java Virtual Machine Program Synthesis JVM Testing Compiler Testing

其他

英语

CPCI-S ; EI

National Natural Science Foundation of China["62002256","61872263"] ; Tianjin Intelligent Manufacturing Special Fund Project[20201180] ; National Science Foundation["CCF-2131943","CCF-2141474"]

Computer Science

Computer Science, Software Engineering ; Computer Science, Theory & Methods

WOS:000832185400092

20222812334110

Java programming language ; Network security ; Program compilers ; Program debugging ; Software testing ; Testing ; Virtual machine

Computer Software, Data Handling and Applications:723 ; Computer Programming:723.1 ; Computer Programming Languages:723.1.1 ; Computer Applications:723.5 ; Numerical Methods:921.6

Web of Science

1.College of Intelligence and Computing, Tianjin University,China
2.Southern University of Science and Technology,China
3.University of Illinois Urbana-Champaign,United States

Yingquan Zhao,Zan Wang,Junjie Chen,et al. History-Driven Test Program Synthesis for JVM Testing[C]. 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA:IEEE COMPUTER SOC,2022:1133-1144.