Memory Assessment of Versatile Video Coding

This paper presents a memory assessment of the next-generation Versatile Video Coding (VVC). The memory analyses are performed adopting as a baseline the state-of-the-art High-Efficiency Video Coding (HEVC). The goal is to offer insights and observations of how critical the memory requirements of VVC are aggravated, compared to HEVC. The adopted methodology consists of two sets of experiments: (1) an overall memory profiling and (2) an inter-prediction specific memory analysis. The results obtained in the memory profiling show that VVC access up to 13.4x more memory than HEVC. Moreover, the inter-prediction module remains (as in HEVC) the most resource-intensive operation in the encoder: 60%-90% of the memory requirements. The inter-prediction specific analysis demonstrates that VVC requires up to 5.3x more memory accesses than HEVC. Furthermore, our analysis indicates that up to 23% of such growth is due to VVC novel-CU sizes (larger than 64x64).

💡 Research Summary

This paper conducts a systematic memory‑access analysis of the next‑generation video codec Versatile Video Coding (VVC) and compares the results with the current state‑of‑the‑art High‑Efficiency Video Coding (HEVC). The authors adopt two complementary experimental approaches: (1) a full‑encoder memory profiling that measures total read and write traffic across all functional modules, and (2) a focused investigation of the inter‑frame prediction stage, which is known to dominate computational load in block‑based codecs.

The experimental platform uses the reference software implementations VTM 8.0 for VVC and HM 16.18 for HEVC. Two encoder configurations are examined – Low‑Delay and Random‑Access – and eight test sequences covering HD 1080p, 2K, and 4K resolutions are processed. For each run, only the first 17 frames are encoded to keep the workload manageable while still exposing the core algorithmic behavior. Memory activity is captured with Intel VTune Amplifier for the overall profiling, while custom trace generation and a dedicated simulator are employed to quantify memory bandwidth consumption during inter‑prediction.

Results from the overall profiling reveal that VVC requires dramatically more memory traffic than HEVC. In Low‑Delay mode, VVC’s memory accesses are 7.4 × to 9.1 × higher; in Random‑Access mode the factor rises to 12.1 × to 13.4 ×. These ratios remain stable across different content types, indicating that the increase is intrinsic to the codec design rather than a side effect of particular video characteristics.

A module‑level breakdown shows that inter‑prediction dominates memory usage in both standards, accounting for 60 %–90 % of total accesses. Within this module, fractional‑precision prediction (INTER‑F) alone consumes 12 %–39 % of the traffic, highlighting the cost of sub‑pixel motion compensation. VVC’s richer set of prediction tools – including a quadtree‑with‑multi‑type‑tree (QT‑MTT) partitioning scheme and a larger set of block sizes – expands the number of candidate blocks that must be fetched and evaluated, thereby inflating memory demand.

The dedicated inter‑prediction analysis confirms these observations. On average, VVC accesses 3.5 × to 5.3 × more memory than HEVC during motion compensation. The highest overhead (5.3 ×) occurs on a 4K “Campfire” sequence under Random‑Access settings, while the lowest (2.6 ×) appears on a low‑motion HD clip. This variation correlates with motion complexity and texture richness, underscoring that content characteristics modulate the memory penalty.

A further breakdown by coding‑unit (CU) size isolates the impact of VVC’s novel large CUs (128×128, 128×64, 64×128). These blocks contribute between 4.1 % and 23.3 % of the total inter‑prediction memory accesses, with the upper bound observed on a 4K “TrafficFlow” sequence. The proportion is higher for high‑resolution content, reflecting that large CUs are more frequently selected when they can capture homogeneous regions efficiently. Nonetheless, their presence introduces a non‑trivial memory overhead that must be addressed in hardware or software optimizations.

The authors conclude that VVC’s superior compression performance comes at the cost of a substantially larger memory footprint, especially in the inter‑prediction stage. This creates a clear research gap: designing memory‑efficient VVC encoders for energy‑constrained platforms such as mobile devices, embedded systems, and edge servers. Potential avenues include cache‑aware data layouts, selective activation of large CUs, early‑termination heuristics for block‑partition search, and hardware accelerators that can stream motion‑compensation data with reduced bandwidth. By quantifying the memory impact of each VVC innovation, the paper provides a valuable baseline for future work aimed at reconciling high compression efficiency with low power consumption.