How Much Progress Has There Been in NVIDIA Datacenter GPUs?

Graphics Processing Units (GPUs) are the state-of-the-art architecture for essential tasks, ranging from rendering 2D/3D graphics to accelerating workloads in supercomputing centers and, of course, Artificial Intelligence (AI). As GPUs continue improving to satisfy ever-increasing performance demands, analyzing past and current progress becomes paramount in determining future constraints on scientific research. This is particularly compelling in the AI domain, where rapid technological advancements and fierce global competition have led the United States to recently implement export control regulations limiting international access to advanced AI chips. For this reason, this paper studies technical progress in NVIDIA datacenter GPUs released from the mid-2000s until today. Specifically, we compile a comprehensive dataset of datacenter NVIDIA GPUs comprising several features, ranging from computational performance to release price. Then, we examine trends in main GPU features and estimate progress indicators for per-memory bandwidth, per-dollar, and per-watt increase rates. Our main results identify doubling times of 1.44 and 1.69 years for FP16 and FP32 operations (without accounting for sparsity benefits), while FP64 doubling times range from 2.06 to 3.79 years. Off-chip memory size and bandwidth grew at slower rates than computing performance, doubling every 3.32 to 3.53 years. The release prices of datacenter GPUs have roughly doubled every 5.1 years, while their power consumption has approximately doubled every 16 years. Finally, we quantify the potential implications of current U.S. export control regulations in terms of the potential performance gaps that would result if implementation were assumed to be complete and successful. We find that recently proposed changes to export controls would shrink the potential performance gap from 23.6x to 3.54x.

💡 Research Summary

The paper provides a systematic quantitative analysis of the technical progress of NVIDIA’s datacenter GPUs from the mid‑2000s through 2025, and links these trends to recent U.S. export‑control policy changes. After a brief architectural overview—highlighting the evolution from early CUDA‑core‑only designs to the inclusion of Tensor Cores, FP16 support, sparsity acceleration, and the shift from GDDR to high‑bandwidth memory (HBM)—the authors assemble a comprehensive dataset covering 12 key specifications for every datacenter GPU released in the period. Two parallel analyses are performed: (1) tracking the “top‑performing” GPU each year, and (2) aggregating all models to capture the broader product line.

Performance metrics are modeled using log‑linear regression to extract compound annual growth rates (CAGR) and doubling times (DT). FP16 and FP32 peak throughput grow at 61 % and 55 % per year respectively, yielding DTs of 1.44 years (FP16) and 1.69 years (FP32)—significantly faster than the classic Moore’s Law cadence. FP64, by contrast, improves more modestly (30 %–48 % per year) with DTs ranging from 2.06 to 3.79 years, indicating a de‑prioritization of double‑precision compute for scientific workloads. Off‑chip memory capacity and bandwidth increase at 22 %–29 % per year (DT ≈ 3.3–3.5 years), lagging behind compute growth and reflecting the AI community’s shift toward compute‑bound rather than memory‑bound workloads.

Economic and energy dimensions show that release prices rise at 14 % per year (DT ≈ 5.1 years) while thermal design power (TDP) grows only 5 % per year (DT ≈ 16 years). Consequently, performance‑per‑dollar and performance‑per‑watt improve steadily, though “top‑performing” GPUs exhibit roughly double the price and power growth rates of the overall lineup, underscoring a premium‑segment premiumization trend.

The policy analysis quantifies the impact of recent U.S. export‑control regulations on AI chip access. Under earlier restrictions, the performance gap between U.S.‑available GPUs and those accessible to sanctioned countries was estimated at 23.6×. The latest rule set, which relaxes limits on certain high‑end models, reduces this gap to 3.54×, indicating a substantial narrowing of the technological divide. The authors argue that this reflects a balancing act between national security concerns and the desire to preserve market leadership for U.S. firms.

Limitations are acknowledged: price and TDP data for some models required extrapolation, sparsity support varies across generations, and the study focuses exclusively on NVIDIA, omitting AMD and Intel competitors. Future work is suggested to employ machine‑learning‑based forecasting, incorporate cross‑vendor comparisons, and explore the implications of emerging memory technologies (e.g., GDDR7, HBM3E).

In sum, NVIDIA’s datacenter GPUs have outpaced Moore’s Law in compute performance, with FP16/FP32 throughput doubling in under two years, while memory, price, and power evolve more slowly. These technical advances have enabled the rapid scaling of AI models, and recent export‑control adjustments have markedly reduced the intended performance barrier for foreign entities. The paper provides a valuable benchmark for researchers, industry planners, and policymakers tracking the trajectory of AI‑centric hardware.