A Data-Driven Dynamic Execution Orchestration Architecture

Domain-specific accelerators deliver exceptional performance on their target workloads through fabrication-time orchestrated datapaths. However, such specialized architectures often exhibit performance fragility when exposed to new kernels or irregular input patterns. In contrast, programmable architectures like FPGAs, CGRAs, and GPUs rely on compile-time orchestration to support a broader range of applications; but they are typically less efficient under irregular or sparse data. Pushing the boundaries of programmable architectures requires designs that can achieve efficiency and high-performance on par with specialized accelerators while retaining the agility of general-purpose architectures. We introduce Canon, a parallel architecture that bridges the gap between specialized and general purpose architectures. Canon exploits data-level and instruction-level parallelism through its novel design. First, it employs a novel dynamic data-driven orchestration mechanism using programmable Finite State Machines (FSMs). These FSMs are programmed at compile time to encode high-level dataflow per state and translate incoming meta-information (e.g., sparse coordinates) into control instructions at runtime. Second, Canon introduces a time-lapsed SIMD execution in which instructions are issued across a row of processing elements over several cycles, creating a staggered pipelined execution. These innovations amortize control overhead, allowing dynamic instruction changes while constructing a continuously evolving dataflow that maximizes parallelism. Experimental evaluation shows that Canon delivers high performance across diverse data-agnostic and data-driven kernels while achieving efficiency comparable to specialized accelerators, yet retaining the flexibility of a general-purpose architecture.

💡 Research Summary

The paper introduces Canon, a novel parallel architecture that bridges the gap between highly specialized accelerators and programmable platforms such as FPGAs, CGRAs, and GPUs. Canon’s design rests on two complementary mechanisms: (1) a row‑wise programmable finite‑state‑machine (FSM) orchestrator and (2) a time‑lapsed SIMD execution model.

During compilation, the compiler analyses the high‑level dataflow of a kernel and generates a bit‑stream that programs the FSMs. Each FSM encodes a set of states representing high‑level operations (e.g., multiply, send, accumulate) and transitions that are triggered by incoming meta‑data such as sparse tensor coordinates or messages from neighboring processing elements (PEs). At runtime, the FSM reacts to this meta‑information, dynamically emitting control instructions for its row of PEs. This hybrid static‑dynamic approach confines the cost of dynamic scheduling to the irregular parts of the workload, while the bulk of execution follows a static, high‑throughput schedule.

The second innovation is the time‑lapsed SIMD model. Instead of broadcasting a single instruction to all PEs simultaneously, Canon injects an instruction into the first PE of a row and lets it propagate downstream over a fixed number of cycles (three in the presented design). Consequently, each PE executes the same instruction sequence but at staggered time steps, operating on different data elements. This staggered propagation amortizes control distribution, reduces the need for a global barrier, and enables deterministic, predictable execution even when the instruction stream changes dynamically.

Hardware-wise, Canon consists of a 2‑D mesh of PEs, each with a three‑stage pipeline (LOAD‑EXECUTE‑COMMIT), a four‑word vector lane, local scratch‑pad memory, and a data memory. Control logic is deliberately minimal inside the PE; all orchestration is performed by the FSMs and the dedicated instruction network. Data movement relies primarily on a circuit‑switched NoC that is statically configured for regular patterns. When irregular dependencies arise, the orchestrator can re‑configure the circuit switches or insert memory accesses on‑the‑fly, eliminating the need for a full packet‑switched NoC and its associated overhead.

The memory system is split into a large static region for immutable data (e.g., model weights) and a smaller scratch‑pad for temporary values, both offering single‑cycle random access. This dual‑segment design supports high data reuse for dense workloads while providing a buffer for irregular, runtime‑determined accesses typical of sparse computations.

Evaluation across ten benchmarks—including dense matrix multiplication, structured and unstructured sparse tensor operations, and graph kernels—shows that Canon achieves throughput comparable to domain‑specific accelerators while maintaining power efficiency close to that of specialized designs. Notably, performance degradation remains under 10 % even when sparsity varies from 5 % to 95 %, a regime where conventional FPGA or CGRA solutions suffer significant slow‑downs.

In summary, Canon demonstrates that by combining programmable FSM‑based dynamic orchestration with a time‑lapsed SIMD execution model, it is possible to dramatically lower control and data‑movement overheads while preserving high parallelism. This architecture offers a compelling blueprint for future programmable accelerators capable of handling both regular and highly irregular workloads with near‑specialized efficiency and full programmability.