Deep Learning Pose Estimation for Multi-Label Recognition of Combined Hyperkinetic Movement Disorders

Hyperkinetic movement disorders (HMDs) such as dystonia, tremor, chorea, myoclonus, and tics are disabling motor manifestations across childhood and adulthood. Their fluctuating, intermittent, and frequently co-occurring expressions hinder clinical recognition and longitudinal monitoring, which remain largely subjective and vulnerable to inter-rater variability. Objective and scalable methods to distinguish overlapping HMD phenotypes from routine clinical videos are still lacking. Here, we developed a pose-based machine-learning framework that converts standard outpatient videos into anatomically meaningful keypoint time series and computes kinematic descriptors spanning statistical, temporal, spectral, and higher-order irregularity-complexity features.

💡 Research Summary

This paper presents a comprehensive, video‑based deep‑learning framework for the objective, scalable detection and multi‑label classification of hyperkinetic movement disorders (HMDs) such as dystonia, tremor, myoclonus, chorea, tics, athetosis, ballismus, and stereotypies. Recognizing that current clinical assessment relies heavily on subjective rating scales and suffers from inter‑rater variability—especially when multiple HMDs co‑occur—the authors develop a pipeline that transforms routine outpatient videos, captured with standard smartphones, into anatomically meaningful 2‑D pose keypoint time series using the state‑of‑the‑art YOLOv8 detector.

From each video, 10‑second windows (the clinically relevant segment length) are extracted based on expert annotations. For every window, a rich set of kinematic descriptors is computed, spanning four families: (i) statistical descriptors (mean, variance, displacement, velocity, acceleration), (ii) temporal dynamics (autocorrelation, entropy, fractal dimension, Lempel‑Ziv complexity), (iii) spectral features (power spectral density, dominant frequency, spectral entropy), and (iv) higher‑order irregularity/complexity metrics. In total, roughly 200 features per window are generated, preserving both global movement magnitude and subtle irregularities characteristic of pathological involuntary motions.

The study cohort comprises 21 patients (age 17–75, 12 females) with isolated or combined movement disorders and four healthy controls. Each participant performed a standardized set of motor tasks (rest, posture holding, finger tapping, arm flexion/extension, reaching, writing) designed to elicit the target HMDs. Two movement‑disorder specialists independently annotated the presence (1), absence (0), or uncertainty (2) of each of the eight HMDs per window; disagreements were resolved by consensus, and uncertain windows were excluded to ensure a clean ground‑truth set.

Machine‑learning models (logistic regression, random forests, XGBoost) were trained in a multi‑label setting, with each HMD treated as an independent binary classifier. At the window level, the models achieved phenotype‑specific F1 scores ranging from 0.81 (tremor) to 0.96 (chorea), demonstrating that short video snippets contain sufficient discriminative information. For patient‑level inference, the authors aggregated window probabilities using the 90th percentile (p90) and applied label‑specific thresholds tuned on a control‑aware training regime. This yielded macro‑average AU‑PRC = 0.821 ± 0.019, macro‑average AUC = 0.830 ± 0.029, and a best Hamming accuracy of 0.764 ± 0.041 across five‑fold cross‑validation. Out of 200 individual label decisions (25 subjects × 8 labels), 28 errors occurred (86 % correct), with near‑perfect performance for distinctive phenotypes such as tics (TP = 3, TN = 22) and dystonia (TP = 20, FN = 1, FP = 1).

Permutation‑based feature importance analysis revealed that displacement‑based descriptors dominate decision making, while rhythmicity (spectral) and higher‑order irregularity (complexity) features contribute selectively, often in anatomically plausible regions (e.g., hand/arm for tremor, facial region for tics). This interpretability aligns with clinical intuition that the magnitude of involuntary movement, its rhythmic pattern, and its irregularity are key discriminators among HMDs.

The authors acknowledge several limitations: the modest sample size and single‑center video acquisition may limit external generalizability; variability in lighting, background, and camera angle, despite standardized positioning, could affect pose estimation accuracy; and the exclusion of “uncertain” annotations may underestimate real‑world ambiguity. Future work is proposed to expand to multi‑center, multi‑ethnic datasets, integrate real‑time feedback for clinicians, and combine video‑derived pose features with wearable sensor data for a multimodal assessment platform.

In conclusion, this study demonstrates that a pose‑based deep‑learning pipeline can reliably and explainably phenotype multiple co‑occurring hyperkinetic movement disorders from routine clinical videos. The approach offers a scalable, objective tool for both screening and longitudinal monitoring, paving the way for tele‑neurology applications, personalized therapeutic decision‑making, and more robust outcome measures in clinical trials.