Intention Detection of Gait Adaptation in Natural Settings

Intention Detection of Gait Adaptation in Natural Settings 1 Ines Domingos Hamlyn Centr e Imperial Colle ge London London, United Kingdom ines.domingos16@imperial.ac.uk Guang-Zhong Y ang Medical Robotics Shanghai Jiao T ong University Shanghai, China gzyang@sjtu.edu.cn Fani Deligianni School of Computing Science University of Glasgow Glasgow , United Kingdom fani.deligianni@glasgo w .ac.uk Abstract —Gait adaptation is an important part of gait anal- ysis and its neuronal origin and dynamics has been studied extensively . In neuror ehabilitation, it is important because it enables neuroplasticity mechanisms and facilitates the restoration of motor function. For this r eason, brain–computer interfaces (BCI) ha ve been build to facilitate neuror ehabilitation. This paper presents a gait adaptation scheme in natural settings. It allows monitoring of subjects in more r ealistic en vironment without the r equirement of specialized equipment such as treadmill and foot pressur e sensors. W e extract gait characteristics based on a single RGB camera whereas wireless EEG signals are mon- itored simultaneously . Based on Regularised Common Spatial Patterns (RCSP) that take into consideration both amplitude and frequency EEG features, we demonstrate that the method can not only successfully detect adaptation steps but it also detect efficiently whether the subject adjust their pace to higher or lower speed. Index T erms —Brain Computer Interface, gait adaptation, common spatial patterns, human motion analysis I . I N T RO D U C T I O N Gait adaptation in volv es the ability to change walking direction and/or speed to av oid obstacles and keep balance. Deficiency in walking adaptation indicates a risk factor of falling in the elderly population or patients with Parkinson or stroke [1]–[3]. Gait adaptation plays an important role in neurorehabilitation since it perturbs neuronal dynamics and al- lows patients to restore motor function. Predictiv e framew orks of gait adaptation may rev eal suitable interventions for an effecti ve gait rehabilitation. Additionally , intention detection of mov ement and gait adaptation is a successful way to integrate a lower limb robotic system in patient’ s rehabilitation. Follo wing a neurological injury , such as stroke or spinal cord injury , the ke y to gait recov ery , is neuroplasticity , which is an acti vity-dependent change in brain structure and function. For example, repetiti ve motion patterns enhance neuronal connections in volv ed in the underlying motor task but they could also trigger suboptimal compensation mechanism [4], [5]. Therefore, the timing of initiating therapeutic exercises and mov ements as well as the content of the ex ercises are of paramount importance. 1 Authors ackno wledge funding from EPSRC EP/R026092/1 and EP/R045178/1 Ethical approval has been obtained from ICREC 18IC4816 Numerous researches ha ve sho wn that auditory rhythm has a deep effect on the motor system. These studies sho w that there is a strong connectivity across cortical, subcortical, and spinal lev els between the auditory and motor systems [6]. Based on these e vidences, some gait adaptation studies focus on auditory rhythms, where patients try to couple heel strikes and pacing tones, improving the gait coordination. Consequently , gait adaptation based on split-zone treadmill exercises and auditory rhythm has sho wn to improve gait symmetry in patients with stroke, cerebral palsy and Parkinson disease [6]–[8] and is an ef fectiv e way to adapt stride frequency and improve gait coordination in people after stroke [7]. Here we study gait adaptation based on a rhythmic tone that alternates between three modes of slow , normal and fast pace. The subjects follow the tone as they walk inside a room without any further restriction. The EEG signal is simultaneously recorded via wireless devices. Contrary to previous studies we do not use a treadmill or specialized equipment, which allows the in vestigation of gait adaptation in more natural settings. W e capture gait characteristics such as heel strikes based on a single RGB camera. Subsequently , we use this information and behavioral analysis of the reaction time to extract gait adaptation steps versus non-gait adaptation steps. W e preprocess the EEG signal based on bandpass filtering and independent component analysis (ICA) to remove motion related artefacts and subsequently the signal is epoched based on right/left heel strikes. Finally , EEG gait adaptation char- acteristics are in vestigated based on three classification prob- lems: i) right versus left gait cycle classification (two classes); ii) adaptation versus non adaptation steps (two classes) and iii) adaptation to higher pace versus adaptation steps towards lower pace versus non adaptation steps (three classes). T o this end, we extract features based on regularized common spatial patterns (RCSP) and instantaneous frequency estimation based on the Hilbert transform. Our results show that we can suc- cessfully discriminate adaptation versus non-adaptation with more than 90% testing accuracy , which corresponds to 0.06 10-fold cross-validated generalization loss. Furthermore, we show that combining amplitude and frequency characteristics of the EEG signal outperforms each of them alone. I I . R E L A T E D W O R K A. BCI in gait r ehabilitation A growing number of studies inv estigates brain activity during human locomotion with EEG data. Preceding studies found that cerebral activity increases during walking or prepa- ration for walking and there is a significant activ ation of the sensorimotor area, during isolated leg or foot movements as well as during gait. It is also belie ved that neuronal acti vity has different functional roles according to the frequency ranges, which provide finer details on which brain network features are important in gait control [9]. Recently , brain–computer interfaces hav e been used as a rehabilitation therapy to restore the motor functions in people with gait impairments. Specifically , there has been a huge interest in the use of BCIs in post-stroke gait therapy [10]. This technology can be used in two different approaches. It can be used to control directly the rehabilitation devices or to provide feedback to the user based on brain activity . The feedback is provided by output of rehabilitation devices, for example, the mov ement of a prosthetic limb, activ ated with brain acti vity . Later , when brain activ ation associated with motor intention is measured the information is extracted and used as a signal to control external devices. For the purpose of BCI, the better neural control signal is found in the range of 8-13 Hz ( µ - rhythm), which is found in the central sensory-motor areas. B. Re gularised Common Spatial P atterns CSP has been used successfully before for feature extraction in gait experiments [11]. CSP is known to be very popular and effecti ve but it is also affected by noise and may ov erfit, especially with small datasets [12]. T o ov ercome these disad- vantages of the CSP method, there has been a vast interest in adding prior information to the CSP learning process, using regularization terms [13]–[15]. The process of adding prior information into the CSP method can be achieved with two distinct manners. It can be done either at the cov ariance matrix estimation or at the level of the objectiv e function, which imposes prior information on the spatial filters [12]. So far , se veral RCSP algorithms were dev eloped; among the most notable, the composite CSP (CCSP), the regularized CSP with generic learning approach and the regularized CSP with diagonal loading. The goal of the composite CCSP , proposed by Kang et al [14], is to perform subject-to-subject transfer , which regularizes the covariance matrix using other subjects’ data. The Regularized CSP with generic learning approach was proposed by Lu et al [13] and aims to regularize the cov ariance matrices using data from other subjects. The Regularized CSP with diagonal loading approach uses the Ledoit and W olf ’ s method to decrease the covariance matrix tow ards the identity matrix [15]. Usually , these approaches are based on the amplitude of the EEG signal and ignore its frequency characteristics. I I I . M E T H O D S A. Experimental Setup and Pr ocedures EEG data w as recorded from six healthy participants ( 27 . 5 ± 7 . 58 years). A 32-channels, g.tec Nautilus, EEG wireless acquisition system with active-electrodes (Ag/AgCl) was used. The system records EEG data at 250Hz along with acceleration data in three axes. The EEG cap was placed accordingly to the 10-20 system. Impedance was measured to ensure a value of less than 30Ω for all participants. Data were recorded with open vibe version 1.3 [12]. Participants were asked to walk according to a musical tone that it was programmed to switch between three modes, slow walking, normal walking and fast walking. The frequency at normal walking was 1.75Hz, whereas at slow walking speed was halved and in fast walking, speed was 1.5 times the normal. The duration of each mode w as estimated to be around 24 ± 6 right/left steps, which expresses a random v ariation of up to six steps. Each mode consisted of 20 trials, which resulted in a total of 60 adaptations randomly permuted. The ov erall experiment lasts about 16 minutes. The stimulus was programmed and displayed with Psychtoolbox-3 [16]–[18]. Adaptation ev ents were send to the EEG acquisition server via TCP/IP communication. A Logitech camera has been also used to record participants at 60 frames per second while they were walking. In order to synchronize the camera recording with the EEG acquisition, each captured frame raised an ev ent that was send to the EEG acquisition server via TCP/IP communication. V ideo capturing and ev ents’ transmission was also implemented with Psychtoolbox-3. B. Gait F eatur es Extraction W e are interested in epoching the EEG signal into segments according to left/right heel strikes. T ow ards this aim, we obtain gait information based on the camera recordings and the acceleration data of the EEG system. T o detect and track 2D coordinates of human joints based on a single RGB camera, we used OpenPose [19]–[21]. This is a state-of-the-art, real-time approach that uses deep neural networks to track the joints of multiple-persons stably . Gait analysis based on a single RGB camera is challenging due to the perspecti ve projection and limited 2D information [22]–[25]. T o extract gait ev ents of right and left heel strikes, we estimate the Euclidean distance between the left and right ankle coordinates in Y -camera axis, assuming that the camera is in a vertical position. Singular spectrum analysis (SSA) has been applied to denoise the signal and improve the detection of peaks that reflect foot contacts. SSA has been also used, successfully , to detect heel strikes based on acceleration data [26], [27]. SSA is based on time-series subsampling to construct a trajectory matrix, the so called Hankel matrix. If w is a time-series, then the trajectory matrix takes the form: Fig. 1. Gait features extraction: The video-based gait signal and the acceleration-based gait signal are plotted in blue and green lines, respectively after SSA processing. The extrema points represent left and right heal strikes, respectively . Adaptation events are represented as vertical lines in green, red and blue colors that represent, normal, fast and slow speed, respectiv ely . W =    w 0 · · · w p − 1 . . . . . . . . . w l − 1 · · · w m − 1    (1) Where p = m − l + 1 , m is the length of w and l is the embedding dimension. The signal is reconstructed from av eraging of a subset of the group elementary matrices of the decomposition of the cov ariance matrix: C W = W W T . Here we process acceleration data that comes with the gtec acquisition system to ensure that the EEG signal and the video timeline is fully synchronized. Acceleration data are processed with Principal Component Analysis (PCA) to derive the dominant signal v ariation, which is due to gait. Singular spectrum analysis (SSA) and peak detection was also utilized to detect heel strikes. An example of gait features’ extraction from both the video and the accelerometer is shown in Figure 1. I V . M OV E M E N T A R TE FA C T R E M O V A L EEG-data acquisition is very sensitiv e to motion artefacts. T o eliminate the influence of motion, we ha ve filter the EEG signal based on a bandpass impulse response (FIR) filter of 3-45Hz. The filter is applied forward and then backward to ensure that phase delays are eliminated. Subsequently , we use independent component analysis (ICA) based on the infomax algorithm to remove the influence of motion components [28]. ICA is a common approach of removing gait-related movement artefacts [2], [29]. It inv olves the extraction of maximally independent components. Motion components are normally identified manually based on their frequency profile and their spatial distribution. Subsequently , they are removed and the EEG signal is reconstructed without their influence. A. F eatur e Extraction based on Common Spatial P atterns Here, we in vestigate gait adaptation by formulating a clas- sification problem of whether a step is an adaptation step or not. W e identify adaptation steps based on the reaction time (R T) between the change of the rhythmic tone and the step to match the average step of the session. Non-adaptation steps are drawn from the middle of the trial to match the number of the adaptation steps. T o extract classification features from the EEG data, we use the Common Spatial Patterns (CSP) algorithm, which extracts spatial filters that maximize the discriminability between two classes [12]. CSP uses spatial filters s that maximize the following equations: J ( s ) = s T X T 1 X 1 s s T X T 2 X 2 s = s T C 1 s s T C 2 s (2) X i denotes the matrix k × n for class i , where k is the number of samples and n is the number of channels. C i is the cov ariance matrix of the EEG signal from class i , assuming a zero mean. This problem is transformed to a standard eigen value prob- lem by noting that it is equi valent to maximizing the follo wing function deriv ed based on the Lagrange method [12]: L ( λ, s ) = s T C 1 s − λ ( s T C 2 s − 1) (3) Since we are looking for the extreme points of the function the deriv ate of L with respect to s is zero and therefore: C T 2 C 1 s = λs (4) 1) Re gularised Common Spatial P atterns (RCSP): Al- though, the CSP filters are an ef ficient way of e xtracting spatial filters that discriminate two classes, they are sensitive to noise and outliers. W e have de vised an automated way of extracting right/left heel strikes that occasionally suffer from erroneous peak detection. T o minimize the influence of these outliers in extracting features based on the CSP algorithm we use regularization. W e adopt the Ledoit and W olf ’ s method, which regularizes the covariance matrix by shrinking it to identity [15]. In other words, the goal is to find a linear combination of the identity matrix, I , and the cov ariance matrix, C i , whose expected quadratic loss is minimum. C ∗ i = (1 − ρ ) C i + ρI (5) This method has been sho wn to be particularly effecti ve in problems where the number of samples is less than the number of dimensions [30], [31]. One of its major advantage is that the shrinkage parameter is estimated automatically based on the intrinsic properties of the signal. 2) Common Spatial P atterns of the Hilbert T ransform: Along with the amplitude information of the EEG signal, we are interested in incorporating instantaneous frequency features extracted based on the Hilbert transform of the pre- processed EEG signal, which can be considered a real v alued process x ( t ) . The Hilbert transform is given by the equation: x h ( t ) = 1 π P V Z ∞ −∞ x ( τ ) t − τ dτ (6) where P V represents the Cauchy principal value of the integral. W e estimate the phase of the signal as: φ ( t ) = arctan = x h ( t ) < x h ( t ) (7) where = x h ( t ) and < x h ( t ) is the imaginary and real part of the Hilbert transform of the EEG signal, respectiv ely . The instantaneous frequency is estimated based on the deriv ativ e of the phase. Subsequently , we apply the common spatial patterns Eq. 5 to the instantaneous frequency to extract frequency based characteristics. 3) F r om two-class to multi-class formulation: W e are in- terested in not only identifying the intention to adapt but also determining whether the adaptation is from a slower to faster pace or vice-versa. Therefore, we device a three-classes classification problem that includes adaptation to higher speed, adaptation to lower speed and non-adaptation. CSP and RCSP are intrinsically two-class methods. T o overcome this problem, we construct three pairs of filters between each combination of the three classes. V . R E S U LT S In this study , the in vestigation of gait and gait adaptation is based on the following EEG classification experiments: • Right vs left gait cycle classification (two classes). • Adaptation vs non adaptation steps (two classes). • Adaptation to higher speed vs. adaptation steps towards lower speed vs. non adaptation steps (three classes). T ABLE I B E HA V I O RA L A N ALY SI S O F A D AP TA T I O N : S T EP S T ATI S T I CS Adaptation T ype Step duration (secs) Adaptation time (secs) Adaptation Steps Slow 0 . 57 ± 0 . 039 1 . 12 ± 0 . 4 1 . 95 ± 0 . 51 Normal 0 . 56 ± 0 . 001 1 . 91 ± 0 . 26 3 . 36 ± 0 . 4 Fast 0 . 46 ± 0 . 031 1 . 66 ± 0 . 056 3 . 63 ± 1 . 16 Fig. 2. T esting accuracy of the classifications results of a) Left versus Right (L/R) steps and b) Adaptation versus non-adaptation (A/NA) steps. W e compare results based on CSP and RCSP feature extraction. The results are shown across different sizes of sliding window (w). Fig. 3. 10-fold cross-validation generalization loss of the classifications results of a) Left versus Right (L/R) steps and b) Adaptation versus non-adaptation (A/N A) steps. W e compare results based on CSP and RCSP feature extraction. The results are shown across different sizes of sliding window (w). T able I summarises the behavioral analysis that includes the estimation of step time for each adaptation type: slow , normal and fast. It also includes the time and number of steps required to adapt from one condition to another . This is also called reaction time (R T) and it is estimated as the time between the change of the rhythmic tone and the time when the step matches the av erage step of the session within the standard deviation limit. W e report a conservati ve R T , since we do not include the sessions where the end of adaption is not detected. The classification results for the right versus left gait cycles and for the adaptation versus non-adaptation steps are sum- marized in Figure 2 and Figure 3. Figure 2 demonstrates the testing accuracy of the classifications results of a) left versus right (L/R) steps and b) adaptation versus non-adaptation (A/N A) steps. W e compare results based on CSP and RCSP feature extraction. The results are shown across different sizes of sliding window (w) with a range from 90 samples to 60 samples. Note that 90 samples correspond to 0.36 seconds, whereas the event duration is taken to be 0.4 seconds. In practice the step size may vary as it is shown in T able I. For the adaptation versus non-adaptation training/testing set we assumed that the first three steps after each change of rhythmic tone are adaptation steps and subsequently we chose three steps at the middle of each adaptation trial as the non- adaptation steps. For classification we used support vector machines (SVM) based on radial basis function but we also show the results based on linear discriminant analysis (LD A). Figure 3 demonstrates the 10-fold cross-validation general- ization loss of the classifications results of a) left versus right (L/R) steps and b) adaptation versus non-adaptation (A/N A) steps. W e compare results based on CSP and RCSP feature extraction. The results are shown across different sizes of sliding window (w), similarly to Figure 2. Figure 5 demonstrates the improvement of classification with the application of common support patterns on both the amplitude and instantaneous frequency characteristics of the Hilbert transform of the signal. In particular , Figure 5a- b) shows the testing accuracy and generalisation loss for each subject, respectiv ely , for Left versus Right (L/R) steps. Figure 5c-d) shows the the testing accuracy and generalisation loss for each subject, respectiv ely , for adaptation versus non- adaptation (A/N A) steps. W e see a consistent improvement of the classification of performance across all subjects when both amplitude and frequency characteristics are incorporated. T able II demonstrates the average confusion matrix (per - centage) across subjects for the three-class classification prob- lem. Feature extraction was based on three RCSP filters for each combination of classes. Classification was performed based on SVM with a linear kernel to avoid o verfitting. Results are shown for a sliding window of 90 samples, which was shown to be most effecti ve. The diagonal elements of the matrix represent the sensitivity results for each class. Note that the summation of the vertical columns results in 100%. T ABLE II C O NF U S I ON M A T R I X ( % ) F O R T H R EE C L AS S C LA S S I FIC ATI O N T rue Class predicted class Adaptation type Slower to faster Faster to slower Non- Adaptation Slower to faster 84 . 79 ± 4 . 66 5 . 08 ± 3 . 32 8 . 69 ± 2 . 49 Faster to slower 6 . 34 ± 3 . 45 86 . 73 ± 3 . 36 8 . 08 ± 3 . 92 Non-adaptation 0 . 86 ± 2 . 85 8 . 17 ± 2 . 32 83 . 22 ± 5 . 03 Figure 4 and Figure 6 demonstrate the analysis of the most significant RCSP components and their spatial distribution, re- spectiv ely . Figure 4 shows the average across subjects, 10-fold cross-validation, generalisation loss as we pick more RCSP components from the most significant to less significant eigen values. W e note that for both type of two-class classification the error drops significantly for the first fi ve components but it remains the same or even increases when more components are incorporated. Figure 6 shows the spatial distribution of the fiv e more significant RCSP components for one of the subjects. The top row sho ws the RCSP filters associated with adaptation/non- adaptation classification, whereas the middle row shows the RCSP filters associated with right/left classification. The bot- tom row shows the spatial distribution of the filters associated with adaptation tow ards higher pace versus adaptation tow ards lower pace. Fig. 4. 10-fold cross-validation generalisation loss across the most significant RCSP components for a sliding window of 90 samples. V I . D I S C U S S I O N Gait adaptation plays a significant role in the ability of humans to walk and maintain their balance. In elderly and people with neurological problems, it is an index of their health progression. Therefore, exoskeletons and assistiv e robotic de- vices should be able to sense and quickly adjust to gait changes [32]. This requires decoding neural signals accurately while people walk in their natural en vironments. Most of the adaptation studies today are based on specialized equipment such as split-zone treadmill, whereas they monitor gait with pressure insoles or reflectiv e markers/multi-camera systems. W e hav e de veloped a framew ork to study gait adaptation in natural settings. The subjects walk in a room follo wing the pace of a tone that changes between three modes of slow , normal and fast pace, randomly . W e record EEG signal wirelessly whereas gait characteristics are extracted based on a single RGB camera. The EEG signal is preprocessed based on a bandpass filter of 3-45 Hz, followed by ICA to identify and remove motion- related artefacts. W e use the extracted gait characteristics to epoch the EEG signal and to formulate three classification problems of intention detection in gait adaptation: i) right versus left step, ii) adaptation steps versus non-adaptation steps and iii) adaptation to higher pace versus adaptation to lower pace versus non-adaptation. Subsequently , we use RCSP to extract EEG features that maximize the discriminability between two classes. RCSP is a regularized version of CSP algorithm that al- lows for automatic regularization of the cov ariance matrices for each class. The extent of regularization depends on the intrinsic properties of the data. Here we note a significant improv ement of our results based on the RCSP filters for all types of classification. Furthermore, we show that incorporat- ing both amplitude and frequency based characteristics can further improv e the outcome of the classifier . CSP/RCSP is intrinsically a two-class feature extraction method. T o extend it to three classes we extract features from three pairs of classes: i) adaptation to higher pace vs non- adaptation, ii) adaptation to lower pace vs non-adaptation and iii) adaptation to lower pace versus adaptation to higher pace. Fig. 5. Application of common support patterns on both the amplitude and instantaneous frequency characteristics of the Hilbert transform of the signal: a) shows the testing accuracy for Left versus Right (L/R) steps, b) shows the generalisation loss for each subject for Left versus Right (L/R) steps, c) shows the the testing accuracy for each subject for adaptation versus non-adaptation (A/N A) steps, and d) shows the generalisation loss for each subject for adaptation versus non-adaptation (A/N A) steps. Finally , we in vestigate the influence of the number of components to the classification accuracy and their spatial distribution. W e note that for the two-class experiments five out of 32 components are enough to achiev e high accuracy . Furthermore, the spatial distribution of the RCSP components seems to be of physiological origin. R E F E R E N C E S [1] J. W agner , S. Makeig, M. Gola, C. Neuper , and G. Muller -Putz, “Distinct beta band oscillatory networks subserving motor and cognitiv e control during gait adaptation, ” Journal of Neur oscience , vol. 36, no. 7, pp. 2212–2226, 2016. [2] J. T . Gwin, K. Gramann, S. Makeig, and D. P . Ferris, “Removal of movement artifact from high-density eeg recorded during walking and running, ” Journal of Neur ophysiology , vol. 103, no. 6, pp. 3526–3534, 2010. [3] D. Martelli, V . V ashista, S. Micera, and S. K. Agrawal, “Direction- dependent adaptation of dynamic gait stability following waist-pull perturbations, ” Ieee Tr ansactions on Neural Systems and Rehabilitation Engineering , vol. 24, no. 12, pp. 1304–1313, 2016. [4] T . A. Jones, “Motor compensation and its effects on neural reorgani- zation after stroke, ” Natur e Revie ws Neur oscience , vol. 18, no. 5, pp. 267–280, 2017. [5] V . R. Edgerton, R. D. de Leon, N. Tillakaratne, M. R. Recktenw ald, J. A. Hodgson, and R. R. Roy , “Use-dependent plasticity in spinal stepping and standing, ” Adv Neur ol , vol. 72, pp. 233–47, 1997. [6] M. H. Thaut and M. Abiru, “Rhythmic auditory stimulation in reha- bilitation of movement disorders: A review of current research, ” Music P er ception , vol. 27, no. 4, pp. 263–269, 2010. [7] M. Roerdink, C. J. C. Lamoth, G. Kwakkel, P . C. W . van Wieringen, and P . J. Beek, “Gait coordination after stroke: Benefits of acoustically paced treadmill walking, ” Physical Therapy , vol. 87, no. 8, pp. 1009– 1022, 2007. [8] T . E. Howe, B. Lovgreen, F . W . J. Cody , V . J. Ashton, and J. A. Oldham, “ Auditory cues can modify the gait of persons with early- stage parkinson’ s disease: a method for enhancing parkinsonian walking performance?” Clinical Rehabilitation , vol. 17, no. 4, pp. 363–367, 2003. [9] M. Seeber , R. Scherer , J. W agner , T . Solis-Escalante, and G. R. Muller- Putz, “High and low gamma eeg oscillations in central sensorimotor areas are con versely modulated during the human gait cycle, ” Neur oim- age , vol. 112, pp. 318–326, 2015. [10] A. Ramos-Murguialday , D. Broetz, M. Rea, L. Laer , O. Y ilmaz, and F . L. B. et al., “Brain-machine interface in chronic stroke rehabilitation: A controlled study , ” Annals of Neur ology , vol. 74, pp. 100–108, 2013. [11] M. H. B. R. O. V . Chiyuan Zhang, Samy Bengio, “Understanding deep learning requires rethinking generalization, ” ICRL , 2017. [12] F . Lotte and C. T . Guan, “Regularizing common spatial patterns to im- prove bci designs: Unified theory and ne w algorithms, ” Ieee T ransactions on Biomedical Engineering , vol. 58, no. 2, pp. 355–362, 2011. [13] H. P . Lu, K. N. Plataniotis, and A. N. V enetsanopoulos, “Regularized common spatial patterns with generic learning for eeg signal classi- fication, ” Annual International Confer ence of the Ieee Engineering in Medicine and Biology Society , vol. 1-20, 2009. [14] H. Kang, Y . Nam, and S. Choi, “Composite common spatial pattern for subject-to-subject transfer, ” IEEE Signal Pr ocessing Letters , vol. 16, pp. 683–686, 2009. [15] O. Ledoit and M. W olf, “ A well-conditioned estimator for large- dimensional covariance matrices, ” J ournal of Multivariate Analysis , vol. 88, no. 2, pp. 365–411, 2004. [16] D. H. Brainard, “The psychophysics toolbox, ” Spatial V ision , vol. 10, no. 4, pp. 433–436, 1997. [17] D. G. Pelli, “The videotoolbox software for visual psychophysics: T ransforming numbers into movies, ” Spatial V ision , vol. 10, no. 4, pp. 437–442, 1997. Fig. 6. RCSP filters for the fiv e most significant components of one of the subjects. a) T op row shows the RCSP filters associated with adaptation/non- adaptation classification, b) Middle row sho ws the RCSP filters associated with right/left classification, c) Bottom row shows the RCSP filters associated with the adaptation to higher versus adaptation to lower speed. [18] M. Kleiner , D. Brainard, and D. Pelli, “What’ s new in psychtoolbox-3?” P er ception , vol. 36, pp. 14–14, 2007. [19] Z. Cao, T . Simon, S. E. W ei, and Y . Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields, ” 30th Ieee Conference on Computer V ision and P attern Recognition (Cvpr 2017) , pp. 1302–1310, 2017. [20] S. E. W ei, V . Ramakrishna, T . Kanade, and Y . Sheikh, “Conv olutional pose machines, ” 2016 Ieee Conference on Computer V ision and P attern Recognition (Cvpr) , pp. 4724–4732, 2016. [21] T . Simon, H. Joo, I. Matthews, and Y . Sheikh, “Hand keypoint detection in single images using multiview bootstrapping, ” 30th Ieee Conference on Computer V ision and P attern Recognition (Cvpr 2017) , pp. 4645– 4653, 2017. [22] X. Gu, F . Deligianni, B. Lo, W . Chen, and G. Y ang, “Markerless gait analysis based on a single RGB camera, ” in 2018 IEEE 15th International Conference on W earable and Implantable Body Sensor Networks (BSN) . Las V e gas, NV : IEEE, Mar . 2018, pp. 42–45. [23] F . Deligianni, Y . Guo, and G.-Z. Y ang, “From Emotions to Mood Disorders: A Survey on Gait Analysis Methodology , ” IEEE Journal of Biomedical and Health Informatics , vol. 23, no. 6, Nov . 2019. [24] Y . Guo, F . Deligianni, X. Gu, and G.-Z. Y ang, “3-D Canonical Pose Estimation and Abnormal Gait Recognition W ith a Single RGB-D Camera, ” IEEE Robotics and Automation Letters , vol. 4, no. 4, Oct. 2019. [25] X. Gu, Y . Guo, F . Deligianni, B. Lo, and G.-Z. Y ang, “Cross-subject and cross-modal transfer for generalized abnormal gait pattern recognition, ” IEEE T ransactions on Neural Networks and Learning Systems , vol. 32, no. 2, pp. 546–560, 2021. [26] D. Jarchi, C. W ong, R. M. Kwasnicki, B. Heller , G. A. T ew , and G. Z. Y ang, “Gait parameter estimation from a miniaturized ear-worn sensor using singular spectrum analysis and longest common subsequence, ” Ieee T ransactions on Biomedical Engineering , vol. 61, no. 4, pp. 1261– 1273, 2014. [27] F . Deligianni, C. W ong, B. Lo, and G. Z. Y ang, “ A fusion framework to estimate plantar ground force distributions and ankle dynamics, ” Information Fusion , vol. 41, pp. 255–263, 2018. [28] S. Makeig, A. J. Bell, T . P . Jung, and T . J. Sejnowski, “Independent component analysis of electroencephalographic data, ” Advances in Neu- ral Information Pr ocessing Systems 8 , vol. 8, pp. 145–151, 1996. [29] K. L. Snyder , J. E. Kline, H. J. Huang, and D. P . Ferris, “Independent component analysis of gait-related movement artifact recorded using eeg electrodes during treadmill walking, ” F r ontiers in Human Neur oscience , vol. 9, 2015. [30] F . Deligianni, G. V aroquaux, B. Thirion, D. J. Sharp, C. Ledig, R. Leech, and D. Rueckert, “ A framew ork for inter-subject prediction of functional connectivity from structural networks, ” Ieee Tr ansactions on Medical Imaging , vol. 32, no. 12, pp. 2200–2214, 2013. [31] F . Deligianni, G. V aroquaux, B. Thirion, E. Robinson, D. J. Sharp, A. D. Edwards, and D. Rueckert, “ A probabilistic framework to infer brain functional connectivity from anatomical connections, ” in Information Pr ocessing in Medical Imaging , G. Sz ´ ekely and H. K. Hahn, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 296–307. [32] R. V arghese, D. Freer , F . Deligianni, J. Liu, and G.-Z. Y ang, “W earable robotics for upper-limb rehabilitation and assistance: a re view on the state-of-the-art, challenges and future research, ” in W ear able technology in medicine and health care . Elsevier (Academic Press), 2018, pp. 23—-69.