ParalESN: Enabling parallel information processing in Reservoir Computing

Reservoir Computing (RC) has established itself as an efficient paradigm for temporal processing. However, its scalability remains severely constrained by (i) the necessity of processing temporal data sequentially and (ii) the prohibitive memory footprint of high-dimensional reservoirs. In this work, we revisit RC through the lens of structured operators and state space modeling to address these limitations, introducing Parallel Echo State Network (ParalESN). ParalESN enables the construction of high-dimensional and efficient reservoirs based on diagonal linear recurrence in the complex space, enabling parallel processing of temporal data. We provide a theoretical analysis demonstrating that ParalESN preserves the Echo State Property and the universality guarantees of traditional Echo State Networks while admitting an equivalent representation of arbitrary linear reservoirs in the complex diagonal form. Empirically, ParalESN matches the predictive accuracy of traditional RC on time series benchmarks, while delivering substantial computational savings. On 1-D pixel-level classification tasks, ParalESN achieves competitive accuracy with fully trainable neural networks while reducing computational costs and energy consumption by orders of magnitude. Overall, ParalESN offers a promising, scalable, and principled pathway for integrating RC within the deep learning landscape.

💡 Research Summary

Reservoir Computing (RC) has long been praised for its ability to harness rich recurrent dynamics while keeping training cheap: a randomly initialized, fixed high‑dimensional “reservoir” processes the input sequence, and only a linear read‑out is trained. However, two practical bottlenecks limit its scalability. First, the recurrent update must be performed sequentially, which prevents parallel execution on modern accelerators. Second, the reservoir’s transition matrix is dense (size N_h × N_h), so memory consumption grows quadratically with the hidden dimension, quickly exhausting GPU/CPU memory for large reservoirs.

The paper introduces Parallel Echo State Network (ParalESN), a novel RC architecture that overcomes both bottlenecks by redesigning the reservoir as a diagonal linear system in the complex domain and by adding a lightweight mixing layer to re‑introduce non‑linearity. The state update for layer ℓ is

 h_t^{(ℓ)} = (1‑τ^{(ℓ)}) h_{t‑1}^{(ℓ)} + τ^{(ℓ)} ·