Time2Vec Transformer for Robust Gesture Recognition from Low-Density sEMG

Accurate and responsive myoelectric prosthesis control typically relies on complex, dense multi-sensor arrays, which limits consumer accessibility. This paper presents a novel, data-efficient deep learning framework designed to achieve precise and accurate control using minimal sensor hardware. Leveraging an external dataset of 8 subjects, our approach implements a hybrid Transformer optimized for sparse, two-channel surface electromyography (sEMG). Unlike standard architectures that use fixed positional encodings, we integrate Time2Vec learnable temporal embeddings to capture the stochastic temporal warping inherent in biological signals. Furthermore, we employ a normalized additive fusion strategy that aligns the latent distributions of spatial and temporal features, preventing the destructive interference common in standard implementations. A two-stage curriculum learning protocol is utilized to ensure robust feature extraction despite data scarcity. The proposed architecture achieves a state-of-the-art multi-subject F1-score of 95.7% $\pm$ 0.20% for a 10-class movement set, statistically outperforming both a standard Transformer with fixed encodings and a recurrent CNN-LSTM model. Architectural optimization reveals that a balanced allocation of model capacity between spatial and temporal dimensions yields the highest stability. Furthermore, while direct transfer to a new unseen subject led to poor accuracy due to domain shifts, a rapid calibration protocol utilizing only two trials per gesture recovered performance from 21.0% $\pm$ 2.98% to 96.9% $\pm$ 0.52%. By validating that high-fidelity temporal embeddings can compensate for low spatial resolution, this work challenges the necessity of high-density sensing. The proposed framework offers a robust, cost-effective blueprint for next-generation prosthetic interfaces capable of rapid personalization.

💡 Research Summary

The paper tackles the longstanding challenge of achieving high‑performance myoelectric control with minimal hardware. While most recent sEMG‑based prosthetic interfaces rely on dense electrode arrays (often 12 – 128 channels) to extract spatial patterns, this work demonstrates that a carefully designed deep learning architecture can compensate for the loss of spatial resolution by exploiting rich temporal information.

The authors introduce a hybrid Transformer model that incorporates learnable Time2Vec embeddings as a replacement for the conventional fixed sinusoidal positional encodings. Time2Vec learns a set of linear and periodic basis functions directly from the data, allowing the model to adapt to the stochastic temporal warping that characterizes biological movements (e.g., variable gesture duration and speed). By embedding Time2Vec within the self‑attention mechanism rather than as a pre‑processing feature, the model aligns temporal context with spatial features more naturally.

To avoid the destructive interference that occurs when a sine wave is added directly to raw sEMG amplitudes, the paper proposes a Normalized Additive Fusion strategy. Both the convolutional spatial features and the Time2Vec temporal vectors are first passed through LayerNorm, then summed element‑wise. This normalization equalizes their statistical scales, preserving the physical meaning of the amplitude while still providing a unified representation for the Transformer encoder.

The architecture consists of three stages: (1) a lightweight 1‑D convolutional stem that extracts local spatial cues from the two channels, (2) a Time2Vec module that generates a learnable temporal vector for each 250 ms window, and (3) a multi‑head self‑attention encoder that processes the combined embeddings. Extensive ablation studies show that the Normalized Additive Fusion outperforms simple concatenation and naïve addition in both convergence stability and final F1‑score.

Training data are drawn from a public Delsys DE‑2.x dataset containing eight healthy subjects performing ten dynamic gestures (five single‑finger flexions, four thumb‑opposition combos, and a gross hand‑close). Each gesture is recorded in six 5‑second trials. The authors segment the recordings with a 250 ms sliding window and 125 ms stride, yielding a refresh rate of 8 Hz, which satisfies the sub‑300 ms latency requirement for responsive prosthetic control.

A rigorous 8‑fold Leave‑One‑Subject‑Out (LOSO) cross‑validation protocol is employed. For each fold, seven subjects are used for multi‑subject training (chronological split: trials 1‑4 for training, trial 5 for validation, trial 6 for evaluation). The held‑out subject is completely unseen during pre‑training. To simulate rapid personalization, the first trial of the held‑out subject is used for fine‑tuning (calibration), the second trial for validation, and the remaining four trials for final testing.

Results:

• Multi‑subject evaluation yields an average F1‑score of 95.7 % ± 0.20 %, surpassing a standard Transformer with fixed sinusoidal encodings (92.3 % ± 0.45 %) and a strong CNN‑LSTM baseline (89.1 % ± 0.62 %).
• Direct transfer to a new user without adaptation drops dramatically to 21.0 % ± 2.98 %, confirming the presence of substantial domain shift.
• After only two calibration trials (≈ 500 ms of data per gesture), fine‑tuning recovers performance to 96.9 % ± 0.52 %, demonstrating that the model can be personalized with negligible user effort.

Model capacity analysis under a fixed FLOPs budget reveals that a balanced allocation of parameters between the convolutional stem and the Transformer encoder (approximately 1:1 ratio) yields the highest stability and accuracy. This finding supports the authors’ hypothesis that, for low‑density sEMG, temporal resolution is as critical as spatial amplitude information.

The paper also positions its contributions relative to prior work. Earlier studies such as CT‑HGR (128‑channel) and TraHGR (12‑channel) achieved respectable accuracies but required dense sensor setups and used standard learnable positional embeddings that lack explicit periodic inductive bias. A previous attempt to use Time2Vec as a pre‑processing step actually degraded performance, likely because it was concatenated at the input rather than integrated into the attention mechanism. By contrast, the present approach demonstrates a 3.2 %p absolute gain over those baselines, confirming the value of learnable temporal embeddings when properly embedded in the Transformer architecture.

In summary, the authors provide a compelling blueprint for cost‑effective, high‑performance myoelectric control:

1. Hardware simplicity – only two sEMG electrodes are required, dramatically reducing device cost, power consumption, and user discomfort.
2. Algorithmic sophistication – Time2Vec embeddings capture subject‑specific periodicities; Normalized Additive Fusion preserves signal integrity; a two‑stage curriculum mitigates data scarcity.
3. Rapid personalization – a calibration protocol with merely two trials per gesture restores near‑perfect accuracy for unseen users, making the system viable for consumer‑grade prostheses.

Future directions suggested include on‑device real‑time deployment, expansion to larger gesture vocabularies, and exploration of meta‑learning or domain‑adaptation techniques to further reduce calibration overhead. The work convincingly shows that high‑quality temporal embeddings can replace dense spatial sensing, opening a path toward accessible, scalable prosthetic interfaces.