Data access optimizations for highly threaded multi-core CPUs with multiple memory controllers

Processor and system architectures that feature multiple memory controllers are prone to show bottlenecks and erratic performance numbers on codes with regular access patterns. Although such effects are well known in the form of cache thrashing and aliasing conflicts, they become more severe when memory access is involved. Using the new Sun UltraSPARC T2 processor as a prototypical multi-core design, we analyze performance patterns in low-level and application benchmarks and show ways to circumvent bottlenecks by careful data layout and padding.

💡 Research Summary

The paper investigates memory‑access bottlenecks that arise on highly threaded multi‑core CPUs equipped with several independent memory controllers, using the Sun UltraSPARC T2 as a representative platform. The T2 features eight cores and eight memory controllers, each attached to a 64‑byte cache line. Because the low‑order bits of a physical address directly select a memory bank, regular access patterns (e.g., sequential arrays aligned on 64‑byte boundaries) can cause many cores to target the same bank simultaneously. This “address aliasing” leads to severe bank contention, cache line thrashing, and highly variable performance, especially as the number of active threads grows.

The authors first present a theoretical model of the address‑to‑bank mapping: bits 0‑5 form the line offset, bits 6‑8 select the bank, and higher bits address rows. They then conduct a systematic experimental study with micro‑benchmarks (STREAM, pointer‑chasing, and a 3‑D stencil kernel) and with a small set of real applications. Using hardware performance counters, they measure per‑controller request rates, cache miss ratios, and overall bandwidth while varying data size, stride, and thread count.

Key observations include:

1. Stride sensitivity – Strides of 64 B or 128 B concentrate traffic on a single bank, reducing effective bandwidth to less than 30 % of peak. Larger strides (≥256 B) spread requests evenly and recover most of the bandwidth.
2. Thread scaling – With an optimal stride, bandwidth scales almost linearly up to eight threads; with a poor stride, scaling stalls after four threads due to controller saturation.
3. Application impact – The 3‑D stencil kernel, which accesses three‑dimensional arrays with complex strides, achieves less than 2× speed‑up on eight threads in the naïve layout, illustrating that real codes suffer even more from aliasing than synthetic tests.

To mitigate these effects, the paper proposes two complementary techniques:

Data padding – Inserting a small amount of extra space (typically 8 B or 16 B) between consecutive elements of an array changes the low‑order address bits, thereby forcing a more uniform distribution across banks. Experiments show that padding can increase sustained bandwidth by a factor of 1.8–2.2 and cut cache miss rates by roughly 40 %.

Stride realignment – Adjusting loop increments to multiples of 8 or 16 elements ensures that successive memory accesses rotate through the banks rather than repeatedly hitting the same one. This can be expressed through compiler pragmas or explicit address calculations. When applied to the stencil kernel, stride realignment combined with padding reduces total execution time by 30 %–45 % and shrinks performance variance (standard deviation) by more than 70 %.

When both techniques are applied together, the eight‑core T2 reaches about 85 % of its theoretical memory bandwidth (≈64 GB/s) and exhibits near‑linear scaling. The authors argue that traditional NUMA‑aware optimizations are insufficient for architectures with multiple controllers; instead, developers must consider the physical address mapping when designing data structures.

The paper concludes with a discussion of future work: (1) automated tools that insert optimal padding based on compiler analysis, (2) runtime monitors that dynamically adapt stride and padding to current bank utilization, and (3) validation of the proposed methods on other multi‑controller CPUs such as AMD EPYC and Intel Xeon Scalable families.

In summary, the study demonstrates that modest, architecture‑aware changes to data layout—specifically padding and stride alignment—can dramatically alleviate memory controller contention on highly threaded multi‑core systems, delivering predictable, high‑throughput performance without requiring major algorithmic redesign.