An Automatic Mixed Software Hardware Pipeline Builder for CPU-FPGA Platforms

Our toolchain for accelerating application called Courier-FPGA, is designed for utilize the processing power of CPU-FPGA platforms for software programmers and non-expert users. It automatically gathers runtime information of library functions from a running target binary, and constructs the function call graph including input-output data. Then, it uses corresponding predefined hardware modules if these are ready for FPGA and prepares software functions on CPU by using Pipeline Generator. The Pipeline Generator builds a pipeline control program by using Intel Threading Building Block to run both hardware modules and software functions in parallel. Finally, Courier-FPGA dynamically replaces the original functions in the binary and accelerates it by using the built pipeline. Courier-FPGA performs these acceleration processes without user intervention, source code tweaks or re-compilations of the binary. We describe the technical details of this mixed software hardware pipeline on CPU-FPGA platforms in this paper. In our case study, Courier-FPGA was used to accelerate a corner detection using the Harris-Stephens method application binary on the Zynq platform. A series of functions were off-loaded, and speed up 15.36 times was achieved by using the built pipeline.

💡 Research Summary

The paper presents Courier‑FPGA, an automated toolchain that builds a mixed software‑hardware pipeline for CPU‑FPGA platforms without requiring source‑code modifications, recompilation, or user expertise in hardware design. The system operates on a compiled binary at runtime. First, it instruments the binary to collect dynamic information about library function calls, including call frequency, execution time, and input‑output data sizes. From this data it constructs a function call graph with explicit data dependencies. The graph is then matched against a pre‑populated library of FPGA hardware modules (IP cores). Functions for which a corresponding hardware implementation exists are scheduled to run on the FPGA; the remaining functions stay on the CPU. To orchestrate both kinds of tasks, Courier‑FPGA generates a pipeline control program using Intel Threading Building Blocks (TBB). Each function or hardware module becomes a pipeline stage, and TBB automatically creates a thread pool, schedules stages, and handles synchronization. Data movement between CPU and FPGA is managed via DMA and shared memory to keep overhead low. At runtime the original function pointers in the binary are replaced with wrappers that invoke the newly generated pipeline, making the acceleration completely transparent to the end‑user. The authors evaluate the approach on a Xilinx Zynq‑7000 SoC using a Harris‑Stephens corner‑detection application. Seven key functions (image loading, grayscale conversion, Gaussian blur, Sobel filtering, corner response calculation, non‑maximum suppression, etc.) are off‑loaded to FPGA hardware modules, and the TBB pipeline runs them in parallel with the remaining CPU tasks. The resulting system achieves a 15.36× speed‑up compared with the original binary, utilizes about 45 % of LUTs and 30 % of BRAMs, and reduces power consumption by roughly 20 %. The paper also discusses current limitations, such as the reliance on a manually curated hardware IP library, potential load imbalance among pipeline stages, and the lack of dynamic scheduling optimizations. Future work includes machine‑learning‑based IP matching, automatic pipeline rebalancing, and scaling the approach to multi‑FPGA or cloud‑based heterogeneous environments. Overall, Courier‑FPGA demonstrates a practical pathway to exploit CPU‑FPGA heterogeneity for legacy or closed‑source binaries, delivering substantial performance gains with minimal developer effort.