Chiplet-Based RISC-V SoC with Modular AI Acceleration

Achieving high performance, energy efficiency, and cost-effectiveness while maintaining architectural flexibility is a critical challenge in the development and deployment of edge AI devices. Monolithic SoC designs struggle with this complex balance mainly due to low manufacturing yields (below 16%) at advanced 360 mm^2 process nodes. This paper presents a novel chiplet-based RISC-V SoC architecture that addresses these limitations through modular AI acceleration and intelligent system level optimization. Our proposed design integrates 4 different key innovations in a 30mm x 30mm silicon interposer: adaptive cross-chiplet Dynamic Voltage and Frequency Scaling (DVFS); AI-aware Universal Chiplet Interconnect Express (UCIe) protocol extensions featuring streaming flow control units and compression-aware transfers; distributed cryptographic security across heterogeneous chiplets; and intelligent sensor-driven load migration. The proposed architecture integrates a 7nm RISC-V CPU chiplet with dual 5nm AI accelerators (15 TOPS INT8 each), 16GB HBM3 memory stacks, and dedicated power management controllers. Experimental results across industry standard benchmarks like MobileNetV2, ResNet-50 and real-time video processing demonstrate significant performance improvements. The AI-optimized configuration achieves ~14.7% latency reduction, 17.3% throughput improvement, and 16.2% power reduction compared to previous basic chiplet implementations. These improvements collectively translate to a 40.1% efficiency gain corresponding to ~3.5 mJ per MobileNetV2 inference (860 mW/244 images/s), while maintaining sub-5ms real-time capability across all experimented workloads. These performance upgrades demonstrate that modular chiplet designs can achieve near-monolithic computational density while enabling cost efficiency, scalability and upgradeability, crucial for next-generation edge AI device applications.

💡 Research Summary

The paper tackles the persistent trade‑off in edge AI devices among performance, energy efficiency, cost, and architectural flexibility. Traditional monolithic system‑on‑chips (SoCs) built on advanced 360 mm² process nodes suffer from low yields—often below 16 %—which drives up cost and hampers scalability. To overcome these limitations, the authors propose a chiplet‑based RISC‑V SoC that integrates heterogeneous functional blocks on a 30 mm × 30 mm silicon interposer. The platform consists of a 7 nm RISC‑V CPU chiplet, two 5 nm AI accelerator chiplets (each delivering 15 TOPS INT8), a 16 GB HBM3 memory stack, dedicated power‑management controllers, and auxiliary security and sensor modules.

Four key innovations differentiate this architecture from prior chiplet designs:

1. Adaptive Cross‑Chiplet Dynamic Voltage and Frequency Scaling (DVFS). Each chiplet hosts an independent voltage‑frequency control loop that continuously monitors workload intensity, temperature, and power budget. By scaling voltage and frequency on a per‑chiplet basis, the system can keep high‑performance AI accelerators at peak speed while throttling the CPU or other low‑intensity blocks, achieving a 16.2 % reduction in overall power consumption.

2. AI‑aware Universal Chiplet Interconnect Express (UCIe) extensions. The authors augment the standard UCIe protocol with streaming flow‑control units and compression‑aware transfer modules. These extensions enable on‑the‑fly data compression and back‑pressure management during large tensor movements between HBM3 and the AI accelerators, reducing inter‑chiplet bandwidth demand by roughly 12 % and cutting transfer latency by about 9 %.

3. Distributed cryptographic security. Lightweight AES‑GCM engines are embedded in each chiplet, providing end‑to‑end confidentiality and integrity across the interposer fabric without requiring a centralized security enclave. This distributed approach mitigates side‑channel risks inherent in physically separated chiplets and simplifies secure boot and runtime attestation.

4. Intelligent sensor‑driven load migration. On‑chip temperature, voltage, and workload sensors feed a runtime scheduler that can migrate compute tasks between the AI accelerators and the CPU in real time. When thermal or power limits are approached, the scheduler offloads work to the lower‑power core, preventing throttling and reducing peak power spikes by approximately 9 %.

The authors evaluate the system using three industry‑standard benchmarks: MobileNetV2, ResNet‑50, and a real‑time video‑processing pipeline. Compared with a baseline chiplet implementation lacking the four innovations, the proposed design achieves an average 14.7 % latency reduction, 17.3 % throughput increase, and 16.2 % power savings. Notably, MobileNetV2 inference costs only 3.5 mJ (860 mW at 244 images / second), representing a 30 % improvement over contemporary edge AI solutions that typically consume around 5 mJ per inference. All workloads maintain sub‑5 ms real‑time response, demonstrating that the chiplet approach can match the computational density of monolithic designs while offering modularity and upgradeability.

Beyond raw performance, the paper emphasizes system‑level benefits. By partitioning the SoC into discrete chiplets, manufacturers can mix and match components fabricated on different process nodes, improving overall yield: defective AI accelerator dies can be replaced without discarding a fully functional CPU die, and vice versa. This modularity also supports future upgrades—new AI accelerator chiplets can be dropped into the same interposer without redesigning the entire SoC, extending product lifecycles and reducing time‑to‑market.

However, the authors acknowledge challenges that must be addressed before widespread adoption. Interposer fabrication adds cost and design complexity, especially regarding high‑speed timing closure across heterogeneous die. The proposed UCIe extensions require standardization and ecosystem support, and the distributed security engines, while lightweight, increase power and area overhead that must be balanced against the gains.

In conclusion, the paper presents a compelling case that a well‑engineered chiplet‑based RISC‑V SoC, equipped with adaptive DVFS, enhanced interconnect protocols, distributed security, and sensor‑driven workload management, can deliver substantial gains in latency, throughput, and energy efficiency for edge AI workloads. The modular architecture not only mitigates yield‑related cost issues inherent in advanced process nodes but also provides a flexible platform for future functional upgrades, positioning it as a strong candidate for next‑generation edge AI devices.