GPU-Augmented OLAP Execution Engine: GPU Offloading

Modern OLAP systems have mitigated I/O bottlenecks via storage-compute separation and columnar layouts, but CPU costs in the execution layer (especially Top-K selection and join probe) are emerging as new bottlenecks at scale. This paper proposes a hybrid architecture that augments existing vectorized execution by selectively offloading only high-impact primitives to the GPU. To reduce data movement, we use key-only transfer (keys and pointers) with late materialization. We further introduce a Risky Gate (risk-aware gating) that triggers offloading only in gain/risk intervals based on input size, transfer, kernel and post-processing costs, and candidate-set complexity (K, M). Using PostgreSQL microbenchmarks and GPU proxy measurements, we observe improved tail latency (P95/P99) under gated offloading compared to always-on GPU offloading. This work extends the risk-aware gating principle used for optimizer-stage GPU-assisted measurement (arXiv:2512.19750) to execution-layer OLAP primitives.

💡 Research Summary

The paper addresses a newly emerging performance bottleneck in modern analytical data warehouses: the CPU cost of execution‑layer operations such as sorting and joining after I/O bottlenecks have been largely eliminated through storage‑compute separation and columnar storage. The authors observe that Top‑K selection (LIMIT‑based ordering) and key‑based join probing become dominant contributors to latency, especially at petabyte‑scale data volumes, and that simply adding more CPU cores does not scale linearly because these operations are inherently data‑intensive and often require O(N log N) work.

To mitigate this, the authors propose a hybrid execution engine that augments, rather than replaces, the existing vectorized execution pipeline. The architecture consists of three components: (1) a classic host CPU that continues to handle data scanning, I/O, and final result assembly; (2) a GPU coprocessor that is invoked only for the high‑impact primitives (Top‑K selection and key‑based join probe); and (3) a “Risky Gate” that decides, on a per‑query basis, whether the GPU path should be taken. The decision is based on a cost model that compares an estimated CPU cost (ˆC_cpu) with a measured or predicted GPU off‑loading cost (ˆC_gpu = transfer + kernel + post‑processing). The gate triggers the GPU only when the difference exceeds a configurable margin, thereby avoiding mis‑offloading for small input sizes where launch and transfer overhead dominate.

A central technical contribution is the “Key‑Only” off‑loading strategy combined with Late Materialization. Because columnar stores keep sort/join keys contiguous, the system extracts only the key values and their RowID pointers, sending this minimal payload to the GPU. The GPU performs parallel selection or hash probing on the key vector and returns a list of sorted pointers. The host then fetches the remaining columns for only those pointers, dramatically reducing host‑to‑device bandwidth consumption. Experiments show a 16.2× reduction in transfer time at N = 3 M rows and up to a 12.9× overall latency improvement when the full pipeline (transfer + kernel + late materialization) is considered.

The evaluation uses PostgreSQL 16 micro‑benchmarks on a workstation equipped with an AMD Ryzen 7 CPU and an NVIDIA RTX 4060 Laptop GPU. Several key findings emerge:

1. Bandwidth Wall – Full‑row transfers become the dominant cost as N grows, while Key‑Only transfers scale linearly and keep the GPU path viable for large datasets.
2. Small‑N Overhead – For small N (e.g., < 20 k rows), GPU launch and transfer overhead outweigh any computational gains; the Risky Gate eliminates this penalty by falling back to the CPU.
3. Gate Margin Sensitivity – Adjusting the margin (e.g., 0 ms, 5 ms, 10 ms) shifts the N threshold at which the GPU is invoked, providing a tunable knob for workload‑specific policies.
4. CPU vs. Top‑K Scaling – Full sort exhibits O(N log N) growth, whereas Top‑K selection grows more modestly, justifying the focus on Top‑K rather than attempting to accelerate full sorts.
5. Break‑Even Analysis – By fitting empirical constants to a CPU full‑sort model (T_cpu = a·N·log N + b) and a Key‑Only transfer model (T_tx = c·N + d), the authors compute a break‑even point N* where CPU and GPU paths are equal. The predicted N* matches measured data within 2.7 % error, confirming the validity of the cost model.

Overall, the paper demonstrates that selective GPU off‑loading, guided by a risk‑aware gating mechanism, can substantially improve tail latency (P95/P99) for OLAP workloads without incurring unnecessary overhead on small queries. The approach is pragmatic: it leverages existing vectorized execution, requires only modest changes to the execution engine (key extraction, pointer handling, and a gating decision module), and can be combined with other accelerators such as FPGAs for complementary tasks (e.g., streaming filters).

In conclusion, the authors provide a concrete, experimentally validated pathway to integrate GPUs into production OLAP systems. Their contributions include (1) a cost‑aware “Risky Gate” that decides when to turn the GPU on, (2) a Key‑Only + Late Materialization pipeline that dramatically reduces data movement, and (3) a thorough performance model that predicts the break‑even point and guides policy tuning. Future work could explore multi‑GPU scheduling, adaptive gate learning, and broader primitive coverage (e.g., complex aggregations), but the current study already offers a solid foundation for making GPU acceleration a practical, latency‑focused component of modern analytical databases.