Parallel Binary Code Analysis

Binary code analysis is widely used to assess a program’s correctness, performance, and provenance. Binary analysis applications often construct control flow graphs, analyze data flow, and use debugging information to understand how machine code relates to source lines, inlined functions, and data types. To date, binary analysis has been single-threaded, which is too slow for applications such as performance analysis and software forensics, where it is becoming common to analyze binaries that are gigabytes in size and in large batches that contain thousands of binaries. This paper describes our design and implementation for accelerating the task of constructing control flow graphs (CFGs) from binaries with multithreading. Existing research focuses on addressing challenging code constructs encountered during constructing CFGs, including functions sharing code, jump table analysis, non-returning functions, and tail calls. However, existing analyses do not consider the complex interactions between concurrent analysis of shared code, making it difficult to extend existing serial algorithms to be parallel. A systematic methodology to guide the design of parallel algorithms is essential. We abstract the task of constructing CFGs as repeated applications of several core CFG operations regarding to creating functions, basic blocks, and edges. We then derive properties among CFG operations, including operation dependency, commutativity, monotonicity. These operation properties guide our design of a new parallel analysis for constructing CFGs. We achieved as much as 25$\times$ speedup for constructing CFGs on 64 hardware threads. Binary analysis applications are significantly accelerated with the new parallel analysis: we achieve 8$\times$ for a performance analysis tool and 7$\times$ for a software forensic tool with 16 hardware threads.

💡 Research Summary

This paper presents a groundbreaking methodology and implementation for parallelizing binary code analysis, specifically targeting the construction of Control Flow Graphs (CFGs), a fundamental and computationally intensive step in understanding executable programs. The driving motivation is the inadequacy of traditional single-threaded analysis tools in the face of modern software scales, such as multi-gigabyte binaries (e.g., from TensorFlow) and batch analysis of thousands of binaries for software forensics.

The core challenge addressed is that existing state-of-the-art serial CFG construction algorithms, which expertly handle complex compiler-generated constructs like shared code regions, jump tables, non-returning functions, and tail calls, were not designed for concurrent execution. Simply running these algorithms in parallel leads to intricate race conditions and data corruption when threads analyze interdependent or shared code sections.

To systematically overcome this, the authors introduce a novel theoretical framework. They abstract the CFG construction process as the repeated application of a small set of primitive operations (e.g., create function, create block, create edge). They then formally define key properties among these operations: dependencies (when one operation’s outcome affects another), commutativity (when the order of two operations does not affect the final CFG state), and monotonicity (the property that the CFG only gains information as analysis progresses). This framework serves a dual purpose: it exposes subtle flaws in existing serial algorithms that arise from unconsidered interactions between different code constructs, and it provides clear guidance for designing correct and efficient parallel algorithms. The goal becomes maximizing the execution of commutative operations in parallel while carefully managing dependencies with minimal synchronization.

Guided by this framework, the researchers designed new parallel algorithms and concurrent data structures. They moved beyond naive function-level parallelism to tackle issues like synchronized analysis of shared code regions, redesigning jump table analysis and tail call identification to be more parallel-friendly, and minimizing the use of heavyweight locking. This new parallel CFG construction engine was implemented within the widely-used Dyninst binary analysis and instrumentation toolkit.

The performance evaluation demonstrates remarkable results. For the core task of CFG construction, speedups of up to 25x were achieved using 64 hardware threads, showing excellent scalability. Crucially, this low-level acceleration translates directly into significant end-to-end speedups for real-world binary analysis applications. The hpcstruct tool from HPCToolkit, which recovers program structure for performance analysis, saw an 8x speedup using 16 threads. The BinFeat software forensics tool, which extracts features from binaries for machine learning, achieved a 7x speedup with 16 threads. These improvements have the potential to transform workflows in performance tuning and security analysis by drastically reducing the time required for binary analysis.

In summary, this work makes four key contributions: 1) A formal framework of CFG operations and their properties for reasoning about correctness and performance in CFG construction. 2) A new parallel CFG construction algorithm derived from this framework. 3) A practical implementation within the Dyninst framework, making it accessible to the community. 4) A demonstration of its transformative impact on two critical application domains, proving that parallel binary analysis is both feasible and highly beneficial for modern large-scale software analysis challenges.