TY - GEN
T1 - Systematic testing of refactoring engines on real software projects
AU - Gligoric, Milos
AU - Behrang, Farnaz
AU - Li, Yilong
AU - Overbey, Jeffrey
AU - Hafiz, Munawar
AU - Marinov, Darko
N1 - Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2013
Y1 - 2013
N2 - Testing refactoring engines is a challenging problem that has gained recent attention in research. Several techniques were proposed to automate generation of programs used as test inputs and to help developers in inspecting test failures. However, these techniques can require substantial effort for writing test generators or finding unique bugs, and do not provide an estimate of how reliable refactoring engines are for refactoring tasks on real software projects. This paper evaluates an end-to-end approach for testing refactoring engines and estimating their reliability by (1) systematically applying refactorings at a large number of places in well-known, open-source projects and collecting failures during refactoring or while trying to compile the refactored projects, (2) clustering failures into a small, manageable number of failure groups, and (3) inspecting failures to identify non-duplicate bugs. By using this approach on the Eclipse refactoring engines for Java and C, we already found and reported 77 new bugs for Java and 43 for C. Despite the seemingly large numbers of bugs, we found these refactoring engines to be relatively reliable, with only 1.4% of refactoring tasks failing for Java and 7.5% for C.
AB - Testing refactoring engines is a challenging problem that has gained recent attention in research. Several techniques were proposed to automate generation of programs used as test inputs and to help developers in inspecting test failures. However, these techniques can require substantial effort for writing test generators or finding unique bugs, and do not provide an estimate of how reliable refactoring engines are for refactoring tasks on real software projects. This paper evaluates an end-to-end approach for testing refactoring engines and estimating their reliability by (1) systematically applying refactorings at a large number of places in well-known, open-source projects and collecting failures during refactoring or while trying to compile the refactored projects, (2) clustering failures into a small, manageable number of failure groups, and (3) inspecting failures to identify non-duplicate bugs. By using this approach on the Eclipse refactoring engines for Java and C, we already found and reported 77 new bugs for Java and 43 for C. Despite the seemingly large numbers of bugs, we found these refactoring engines to be relatively reliable, with only 1.4% of refactoring tasks failing for Java and 7.5% for C.
KW - Refactoring engines
KW - Systematic testing
KW - Test clustering
UR - http://www.scopus.com/inward/record.url?scp=84880017980&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84880017980&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-39038-8_26
DO - 10.1007/978-3-642-39038-8_26
M3 - Conference contribution
AN - SCOPUS:84880017980
SN - 9783642390371
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 629
EP - 653
BT - ECOOP 2013, Object-Oriented Programming - 27th European Conference, Proceedings
PB - Springer
T2 - 27th European Conference on Object-Oriented Programming, ECOOP 2013
Y2 - 1 July 2013 through 5 July 2013
ER -