Execution-Centric Characterization of FP8 Matrix Cores, Asynchronous Execution, and Structured Sparsity on AMD MI300A

The AMD MI300A APU integrates CDNA3 GPUs with high-bandwidth memory and advanced accelerator features: FP8 matrix cores, asynchronous compute engines (ACE), and 2:4 structured sparsity. These capabilities are increasingly relied upon by modern HPC and HPC-AI workloads, yet their execution characteristics and system-level implications remain insufficiently understood. In this paper, we present an execution-centric characterization of FP8 matrix execution, ACE concurrency, and structured sparsity on MI300A using targeted microbenchmarks. We quantify occupancy thresholds, fairness, throughput trade-offs under concurrent execution, and context-dependent sparsity benefits. We evaluate representative case studies - transformer-style, concurrent, and mixed-precision kernels - to show how these effects translate into application-level performance and predictability. Our results provide practical guidance for occupancy-aware scheduling, concurrency decisions, and sparsity enablement on MI300A-class unified nodes.

💡 Research Summary

The paper presents an execution‑centric characterization of three advanced accelerator features in AMD’s MI300A APU: native FP8 matrix cores, Asynchronous Compute Engines (ACE), and 2:4 structured sparsity. While prior work has focused on the unified memory subsystem of MI300A, this study isolates the actual compute pipelines by building low‑level HIP micro‑benchmarks that directly invoke MFMA (Matrix Fused Multiply‑Add) instructions. Experiments are performed on a single MI300A node running Linux RHEL 8.10, ROCm 7.2, and the hipcc compiler targeting gfx942. The authors control CPU‑GPU placement, pin cores, and use deterministic compilation flags to reduce variability.

FP8 Matrix Core Characterization
The authors design a minimal kernel that executes a single MFMA tile per launch (16 × 16 × 32 for FP8, with FP32 accumulation). By varying the number of active wavefronts (1 – 256) they map throughput versus occupancy for five precisions (FP64, FP32, FP16, BF16, FP8). The results show that all precisions suffer severe under‑utilization at low occupancy; FP8 reaches >90 % of its theoretical peak only when at least 32 wavefronts are active. Throughput scales sub‑linearly, indicating contention for shared scheduling resources. Additionally, matrix aspect‑ratio experiments reveal that near‑square tiles (M≈N) achieve the highest efficiency, while highly rectangular shapes incur a ~10 % penalty due to wavefront redistribution and less optimal LDS usage.

Asynchronous Compute Engines (ACE) Analysis
ACE enables multiple HSA queues to be mapped to hardware command processors, allowing overlapping kernel execution. The authors launch concurrent GEMM kernels of different precisions (FP8, FP16, FP32) across 1‑6 streams. Overlap efficiency (the fraction of total time during which more than one kernel runs) stays above 85 % for two streams but drops sharply to ≈70 % when four or more streams contend for the same MFMA units and memory bandwidth. Fairness is quantified as 1 – (tmax – tmin)/tmean, yielding values between 0.85 and 0.92, indicating reasonably balanced progress. However, when FP8 and FP32 kernels share an ACE, the FP8 workload tends to dominate MFMA allocation, causing the FP32 stream to lag—an important observation for mixed‑precision pipelines.

Structured Sparsity (2:4) Evaluation
MI300A’s hardware supports 2:4 structured sparsity, where two out of every four elements are zero. The authors implement sparse MFMA kernels that encode the sparsity pattern in metadata. For large matrices (≥ 4096 × 4096) the sparse kernels achieve 1.8 – 2.0 × speed‑up over dense equivalents, confirming the theoretical 2× reduction in arithmetic. For smaller matrices (≤ 1024) the overhead of loading sparsity metadata and padding outweighs the compute savings, resulting in a break‑even point around 2048 × 2048. This demonstrates that sparsity benefits are highly size‑dependent.

Application‑Level Case Studies
Three representative workloads are used to translate micro‑benchmark findings to real‑world performance:

1. Transformer‑style attention – By enabling FP8 MFMA and 2:4 sparsity on the attention matrix, the authors observe a 1.5× overall training speed‑up compared with dense FP16 baseline.
2. Concurrent pipeline – A data‑pre‑processing kernel (FP32) and an inference kernel (FP8) are dispatched to separate ACE queues. Overlap reduces end‑to‑end latency by ~20 % while maintaining fairness above 0.9.
3. Mixed‑precision training – Combining FP8 matrix multiplication with FP16 accumulation, and scheduling kernels to respect the 32‑wavefront occupancy threshold, yields 1.3 – 1.7× throughput gains.

Implications for System Software
The authors synthesize their measurements into concrete recommendations for runtime and scheduler design:

• Enforce a minimum occupancy of ~32 wavefronts for FP8 kernels; adjust block sizes or launch multiple independent GEMMs to meet this threshold.
• Limit concurrent ACE streams to three for FP8‑heavy workloads to avoid MFMA contention; dynamically monitor memory bandwidth utilization to decide when to serialize.
• Apply 2:4 sparsity only when matrix dimensions exceed the identified break‑even point; otherwise fall back to dense execution.
• Incorporate occupancy, aspect‑ratio, and sparsity metadata into the scheduler’s cost model, enabling dynamic placement that can improve overall system efficiency by 15 %–20 %.

The paper concludes by releasing the micro‑benchmark suite (upon acceptance) and calls for further research into compiler‑level transformations that can automatically respect the identified execution constraints. Overall, the work provides the first detailed, execution‑focused view of FP8, ACE, and structured sparsity on MI300A, offering actionable insights for both hardware architects and software developers targeting next‑generation HPC‑AI systems.