Much effort is spent everyday by programmers in trying to reduce long, failing execution traces to the cause of the error. We present a new algorithm for error cause localization based on a reduction to the maximal satisfiability problem (MAX-SAT), which asks what is the maximum number of clauses of a Boolean formula that can be simultaneously satisfied by an assignment. At an intuitive level, our algorithm takes as input a program and a failing test, and comprises the following three steps. First, using symbolic execution, we encode a trace of a program as a Boolean trace formula which is satisfiable iff the trace is feasible. Second, for a failing program execution (e.g., one that violates an assertion or a post-condition), we construct an unsatisfiable formula by taking the trace formula and additionally asserting that the input is the failing test and that the assertion condition does hold at the end. Third, using MAX-SAT, we find a maximal set of clauses in this formula that can be satisfied together, and output the complement set as a potential cause of the error. We have implemented our algorithm in a tool called bug-assist for C programs. We demonstrate the surprising effectiveness of the tool on a set of benchmark examples with injected faults, and show that in most cases, bug-assist can quickly and precisely isolate the exact few lines of code whose change eliminates the error. We also demonstrate how our algorithm can be modified to automatically suggest fixes for common classes of errors such as off-by-one.
A large part of the development cycle is spent in debugging, where the programmer looks at a long, failing, trace and tries to localize the problem to a few lines of source code that elucidate the cause of the problem. We describe a novel algorithm for fault localization for software. The input to our algorithm is a program, a correctness specification (either a post-condition, an assertion, or a "golden output"), and a program input and corresponding execution (called the failing execution) that demonstrates the violation of the specification. The output is a minimal set of program statements such that there exists a way to replace these statements such that the failing execution is infeasible.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
Copyright c ACM [to be supplied]. . . $10.00 Internally, our algorithm uses symbolic analysis of software based on Boolean satisfiability, and reduces the problem to maximum Boolean satisfiability. It takes as input a program and a failing test case and performs the following three steps. First, it constructs a symbolic trace formula for the program path executed by the test input. This is a Boolean formula in conjunctive normal form such that the formula is satisfiable iff the program execution is feasible (and every satisfiable assignment to the formula correspond to the sequence of states in a program execution). The trace formula construction proceeds identically to symbolic execution or bounded model checking algorithms [2,6,11,16].
Second, it extends the trace formula by conjoining it with constraints that ensure the initial state satisfies the values of the failing test and the final states satisfy the program post-condition that was failed by the test. The extended trace formula essentially states that starting from the test input and executing the program trace leads to a state satisfying the post-condition. Obviously, the extended trace formula for a failing execution must be unsatisfiable.
Third, it feeds the extended trace formula to a maximum satisfiability solver. Maximum satisfiability (MAX-SAT) is the problem of determining the maximum number of clauses of a given Boolean formula that can be satisfied by any given assignment. Our tool computes a maximal set of clauses of the extended trace formula that can be satisfied, and take the complement of this set as a candidate set of clauses that can be changed to make the entire formula satisfiable. Since each clause in the extended trace formula can be mapped back to a statement in the code, this identifies a candidate localization of the error in terms of program statements. Note that there may be several minimal sets of clauses that can be found in this way, and we enumerate each minimal set as candidate localizations for the user. In our experiments, we have found that the number of minimal sets enumerated in this way remains small. More precisely, our algorithm uses a solver for partial MAX-SAT. In partial MAX-SAT, the input clauses can be marked hard or soft, and the MAX-SAT instance finds the maximum number of soft clauses that can be satisfied by an assignment which satisfies every hard clause. In our algorithm, we mark the input constraints (that ensure that the input is a failing test) as well as the postcondition are hard. This is necessary: otherwise, the MAX-SAT algorithm can trivially return that changing an input or changing the post-condition can eliminate the failing execution. In addition, in our implementation, we group clauses arising out of the same program statement together, and keep the resulting MAX-SAT instance small.
We have implemented our algorithm in a tool called BugAssist for fault localization of C programs. 1 BugAssist takes as input a C program with an assertion, and a set of failing test cases, and returns a set of program instructions whose replacement can remove the failures. It builds on the CBMC bounded model checker for construction of the trace formula and an off-the-shelf MAX-SAT solver [21] to compute the maximal set of satisfied clauses. We demonstrate the effectiveness of BugAssist on 5 programs from Siemens set of benchmarks with injected faults [8]. The TCAS program in the testsuite is run with all the faulty versions in detail to illustrate the completness of the tool. In each case, we show that BugAssist can efficiently and precisely determine the exact (to the human) lines of code that form the “bug”. The other 4 programs are used to show the scalability of the tool by using error trace reduction methods for real world programs.
We can extend our algorithm to suggest fixes for bugs automatically,
This content is AI-processed based on open access ArXiv data.
