Hand Gesture Recognition from Doppler Radar Signals Using Echo State Networks

Hand gesture recognition (HGR) is a fundamental technology in human computer interaction (HCI).In particular, HGR based on Doppler radar signals is suited for in-vehicle interfaces and robotic systems, necessitating lightweight and computationally efficient recognition techniques. However, conventional deep learning-based methods still suffer from high computational costs. To address this issue, we propose an Echo State Network (ESN) approach for radar-based HGR, using frequency-modulated-continuous-wave (FMCW) radar signals. Raw radar data is first converted into feature maps, such as range-time and Doppler-time maps, which are then fed into one or more recurrent neural network-based reservoirs. The obtained reservoir states are processed by readout classifiers, including ridge regression, support vector machines, and random forests. Comparative experiments demonstrate that our method outperforms existing approaches on an 11-class HGR task using the Soli dataset and surpasses existing deep learning models on a 4-class HGR task using the Dop-NET dataset. The results indicate that parallel processing using multi-reservoir ESNs are effective for recognizing temporal patterns from the multiple different feature maps in the time-space and time-frequency domains. Our ESN approaches achieve high recognition performance with low computational cost in HGR, showing great potential for more advanced HCI technologies, especially in resource-constrained environments.

💡 Research Summary

The paper addresses the growing demand for lightweight, real‑time hand‑gesture recognition (HGR) systems that can operate under the strict power and computational constraints of in‑vehicle interfaces, robotics, and other edge‑computing scenarios. While radar‑based HGR, especially using frequency‑modulated continuous‑wave (FMCW) millimeter‑wave radars, offers robustness to lighting conditions and the ability to capture fine‑grained range and velocity information, existing deep‑learning approaches (e.g., CNN‑LSTM, ResNet‑18) still require substantial GPU resources for both training and inference. To overcome this bottleneck, the authors propose a novel framework built around Echo State Networks (ESNs), a type of reservoir computing (RC) architecture that eliminates the need for back‑propagation through time by keeping the recurrent reservoir weights fixed and training only a linear readout layer.

Signal preprocessing
Raw radar returns are first transformed into range‑Doppler maps (RDMs) via a two‑stage FFT (range FFT on each chirp, Doppler FFT across chirps). The authors then decompose the RDM sequence into two complementary feature maps: a range‑time map (RTM) that aggregates Doppler bins to emphasize spatial dynamics, and a Doppler‑time map (DTM) that aggregates range bins to highlight velocity dynamics. For the Soli dataset, which provides four antenna channels, this yields eight distinct maps per gesture (four RTMs and four DTMs). For the Dop‑NET dataset, only micro‑Doppler maps (MDMs) are available, so a single‑channel ESN is used.

Multi‑reservoir ESN architecture
A key insight is that concatenating heterogeneous feature maps into a single reservoir can cause interference, degrading temporal pattern extraction. To mitigate this, each feature map is fed into its own small ESN reservoir. All reservoirs run in parallel, each with its own random recurrent weight matrix (W_res) and input matrix (W_in). The leaky‑integrator update rule x(t+1) = (1‑α)x(t) + α·tanh(W_in·u(t+1) + W_res·x(t)) is employed, with α (leak rate) and spectral radius ρ (scaled near but below 1) tuned to balance memory depth and stability. After processing the full sequence, the final state vectors from all reservoirs are concatenated into a unified high‑dimensional vector r ∈ ℝ^{M·N}, where M is the number of feature maps and N the number of nodes per reservoir. This design preserves modality‑specific temporal dynamics while keeping each reservoir computationally cheap; the parallel combination yields a rich representation without the quadratic cost of a single large reservoir.

Readout classifiers
Four readout strategies are evaluated on the concatenated state vector r:

1. Ridge Regression (RR L) – a linear classifier with L2 regularization (λ = 0.1), solved analytically for fast training.
2. Non‑linear Ridge Regression (RR N) – applies a feature expansion Ψ(r) =