FaultTracer: A spectrum-based approach to localizing failure-inducing program edits

Detecting faults in evolving systems is important. Change impact analysis has been shown to be effective for finding faults during software evolution. For example, Chianti represents program edits as atomic changes, selects affected tests, and determines a subset of affecting changes that might have caused test failures. However, the number of affecting changes related to each test failure in practice may still be overwhelming for manual inspection. In this paper, we present a novel approach, FaultTracer, which ranks program edits according to their suspiciousness to reduce developer effort in manually inspecting affecting changes. FaultTracer adapts spectrum-based fault localization techniques, which assume the statements that are primarily executed by failed tests are more suspicious, and applies them in tandem with an enhanced change impact analysis to identify failure-inducing edits more precisely. We conducted an experimental study using 23 real versions of four real-world Java programs from the Software Infrastructure Repository. The experimental results show that FaultTracer localizes a real regression fault within top three atomic changes for 14 out of 22 studied real failures. When ranking only method-level changes, compared to the existing ranking heuristic, FaultTracer reduces the number of changes to be manually inspected by more than 50% on the data set of real regression faults, and by more than 60% on the data set of seeded faults. The fault localization component of FaultTracer is 80% more effective than traditional spectrum-based fault localization, and enables similar benefits when using either our enhanced change impact analysis or Chianti. The runtime overhead for FaultTracer to collect extended call graphs is 49.83 s for each subject on average and is only 8.26% more than that for Chianti to collect traditional call graph information.