Euclidean Alignment for Transfer Learning in Multi-band Common Spatial Pattern

Abstract

Stroke is a major cause of death and disability worldwide. Motor-Imagery based Brain-Computer Interface (MI-BCI) models offer a post-stroke rehabilitation option. Existing studies for MI-BCI use Transfer Learning techniques like Euclidean Alignment (EA) but lose important brain information due to bandpass filtering. This study introduces new BCI architecture with multi-band temporal filters and EA. The methods considered here are Filter Bank (FB), Empirical Mode Decomposition (EMD), and Continuous Wavelet Transform (CWT). Results show performance improvements, especially with EA being applied before Filter Bank. These models offer promise for post-stroke rehabilitation, particularly when using EA before the multi-band filter.

1 Introduction

Stroke is a leading cause of disability and mortality worldwide, demands urgent attention due to its deep impact on individuals and healthcare systems [18]. Over the years, significant strides have been made in stroke treatment, revolutionizing patient care and outcomes. One of the most recent methods of treatment is the utilization of Brain-Computer Interface models.

Brain-Computer Interface (BCI) is a system that allows for direct communication between the brain and an electronic device without peripheral muscles [3]. It translates the electrical signal of the brain in a task to a computational system. Invasive and non-invasive acquisition methods can collect the brain’s signal. The invasive ways have a better signal-to-noise ratio (SNR), being between 10 to 100 times better than non-invasive methods, but are more dangerous for the subject, since they carry risks such as infection, tissue damage, and implant rejection due to their surgical nature. The non-invasive equipment is safer than the invasive ones, but they have a more imprecise signal. Electroencephalography (EEG) is the most used equipment for BCI as it is not invasive, has a satisfactory temporal resolution in the range of milliseconds (same as the invasive ones), and is portable, light, and 10 times cheaper than the invasive ones [2].

The BCI models are sensitive to the subject who provided the brain’s signals to train the model. Due to the non-stationary behavior of the EEG signals and the anatomic difference between people’s brains, the model is usually subject-dependent. Transfer Learning (TL) techniques can decrease the signal difference between subjects and reduce that problem. This way, we can use data from a set of subjects (source domain) to train a model to another one (target domain). Riemannian Alignment (RA) and Euclidean Alignments (EA) are the most used Transfer Learning methods [7]. However, those models only use a single band in the temporal filtering step, losing some temporal patterns. In this case, the alignment transforms the signals of the source and the target subjects to the same space.

We propose different ways to use multi-band models with EA, and the evaluated temporal filtering was Filter Bank (FB), Empirical Mode Decomposition (EMD), and Continuous Wavelet Transform (CWT). Using the PhysionetMI dataset, a well-known dataset from literature, the results obtained pointed out that multi-band with EA improves the results of BCI classifiers in this scenario. This is an advantage for the BCI models dedicated to post-stroke motor rehabilitation, as it reduces the calibration time and makes the procedure straightforward to be used with any patient.

2 Related Work

The variations in EEG signals between individuals are remarkable, reflecting the distinct nuances of their brain activities [12]. This condition is intensified in individuals who have suffered some form of stroke. As a result, these differences significantly impact the training phase in BCI, requiring a personalized approach. Therefore, in most cases, it is crucial to train a distinct model for each person. To deal with this situation, transfer learning methods were proposed to use many subjects in a single training step. One of the most used methods is the Riemannian Alignment (RA), which is a method to align the covariance matrix of the signals from each subject during their resting state. However, Riemannian space makes the model more computationally expensive and limited since the classifiers must perform efficiently in that space. To outperform that problem, the Euclidean Alignment (EA) was proposed [7]. EA aligns the EEG trials for each subject in Euclidean space using the mean of the covariance matrix of the subject’s trials as a reference matrix. Thereafter, the transformed reference matrix regarding each subject becomes the identity matrix. This technique reduces the difference between data of distinct subjects, similarly than RA, but without transferring the data to Riemannian space. Therefore, it is possible to use non-Riemannian classifiers, such as Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM) [14, 15]. Moreover, the computational cost is reduced by operating EA instead of RA. When comparing models, EA points out better results than RA when transfer learning is used for BCI applications [7].

After its release, different approaches were proposed to improve EA. For instance, the unification of EA and RA alignment methods using a hybrid (Euclidean and Riemannian) space [12]. In addition, a selection of the subjects with data similar to that of the target one can be considered. [12]. Another approach to improve EA is weighting the subject importance based on the distance between the source and target data [5]. Comparisons of transfer learning methods applied to deep learning classifiers pointed out that RA and EA present better results than normalization techniques [13, 21, 22]. Those articles do not use EA with multi-band techniques. In this work, we propose novel pipelines for EA using multi-bands and parallel application of EA after signal decomposition.

3 Brain-Computer Interfaces for Post-Stroke Motor Rehabilitation

Stroke is a medical emergency that leaves sequels, including cognitive and motor deficits and oropharynxes dysphagia [10]. The rehabilitation of post-stroke patients has great importance to the recovery of their cognitive and motor functions. Also, BCI for post-stroke rehabilitation is an alternative to induce neural plasticity. BCIs support this process by delivering targeted stimuli that create a closed loop for neural reorganization. Engaging directly with external devices activates brain regions linked to movement, strengthening pertinent neural pathways and fostering positive plasticity.

Reinforcement learning promotes BCI for post-stroke motor rehabilitation, since it involves adapting behavior based on the consequences of actions. Within BCIs, individuals can receive instant and personalized feedback corresponding to their rehabilitation endeavors. This feedback creates a positive reinforcement, fortifying the neural connections linked to targeted movement patterns. Furthermore, BCIs aim to close the motor planning loop, aiding in the reconnection of brain circuits responsible for motor control. By converting brain signals into actionable commands for external devices, BCIs support the rehabilitation of motor planning, promoting iterative practice and the gradual enhancement of motor skills.

Furthermore, studies on upper-limb rehabilitation using BCI indicate that the accuracy of the BCI system can improve motor function. Other studies also showed a relationship between the BCI accuracy and clinical gains [2, 4, 16]. This relationship is based on the hypothesis that higher BCI accuracy may improve confidence and motivation in patients, which may better promote plasticity [16]. Moreover, another reason for the success of BCI rehabilitation can be the correlation between engagement and higher levels of patient attention.

Figure 1 represents the BCI cycle for post-stroke motor rehabilitation. First, brain signals are collected through sensors, such as electroencephalogram (EEG). The collected signals are filtered to remove interference and increase the signal-to-noise ratio. Techniques such as temporal and spatial filtering can be applied to extract relevant information. Then, feature extraction and selection steps are performed, and the chosen features are used to classify the signal. With the signals already classified, the system executes commands based on the label. For post-stroke motor rehabilitation, the feedback is usually presented as an image on a screen and an electrical stimulus in the muscle. In the context of BCI systems for patient rehabilitation, it is relevant to mention that a smaller number of electrodes brings benefits in terms of ease of treatment and patient comfort [20]. Furthermore, data alignment is emphasized for synchronizing stimuli and their responses and ensuring more accurate temporal analysis. Data Alignment also proves to be effective in enhancing system efficiency and reducing interference and signal noise. By utilizing Data Alignment, we can reduce the number of trials needed to train the model, resulting in faster and less exhausting treatment.

Fig. 1.

BCI cycle for post-stroke motor rehabilitation.

4 Methods

This section presents the methods from the literature used as a baseline to develop the proposed BCI pipelines. We describe EA, the temporal filter methods used in the experiments, and the Common Spatial Pattern.

4.1 Euclidean Alignment

Euclidean Alignment (EA) was proposed as an alternative to Riemannian Alignment (RA) [7]. The advantages of EA are the reduction of the alignment time when compared to RA, and the maintenance of the data in the Euclidean space. EA uses a reference matrix, calculated through the EEG trials, to align the data from different subjects. A reference matrix $\bar{R}_s$ is the mean of the trial’s covariance for the s-th subject, which is defined by

$$\begin{aligned} \bar{R}_{s} = \frac{1}{N_s}\sum ^{N_s}_{n=1}X^{n}_{s}(X^{n}_{s})^{\text {T}} \end{aligned}$$

(1)

where $N_s$ is the number of trials of subject s, $X_s^n$ is the n-th trial of subject s. The matrix $\bar{R}_{s}$ reduces the difference between the subjects as

$$\begin{aligned} E^{n}_{s}=\bar{R}^{-\frac{1}{2}}_{s} \times X^{n}_{s} \end{aligned}$$

(2)

where $E^n_s$ is the signal of subject s after alignment. The new reference matrix for all the transformed subjects is equal to the identity matrix. Therefore, the new space is the same for all the subjects.

4.2 Temporal Filtering

Temporal filtering is a technique used in data processing to extract or manipulate temporal information in data sets. Its purpose is to enhance signal quality by improving the signal-to-noise ratio. Temporal filtering removes or attenuates unwanted temporal components in the data, allowing the identification of more significant patterns or the reduction of noise. The choice of a temporal filtering technique relies on the particular context of the problem and the goals of the analysis. This section presents four temporal filters: Bandpass, Filter Bank (FB), Empirical Mode Decomposition (EMD), and Continuous Wavelet (CWT).

Bandpass and Filter Bank: Bandpass Filters are widely employed in various signal-processing applications, including wireless communication systems, audio and video processing, and EEG. Then, they are useful for isolating signals of specific frequencies and eliminating unwanted noise [1]. A bandpass filter allows for a specific range of frequencies to pass through the filter while attenuating frequencies outside that range.

A Filter Bank is a group of bandpass filters that divide an input signal into sub-bands. Figure 2 shows a diagram of the Analysis Filter Bank.

Fig. 2.

Filter Bank.

Empirical Mode Decomposition: Empirical Mode Decomposition [9] is a method that decomposes non-stationary and non-linear data in different components to extract information that is not explicit in the original signal. The core of this method lies in the empirical identification of the oscillatory modes inherent in the data, distinguished by their characteristic time scales. After this identification, the data is decomposed accordingly.

The method uses the sifting process [9] to decompose the data into Intrinsic Mode Functions (IMFs) and a residual. Figure 3 presents a schematic representation of an EMD algorithm, with 3 IMFs and a residual as outputs. The sifting process of an x signal consists of the following steps:

• The local extrema of the signal are identified, and all the local maxima and minima are connected by a cubic spline line as the upper envelope. Their mean is denominated $m_{i}$, and the difference between the data and $m_{i}$ is the first component $h_{i}$.

• If the component $h_{i}$ is an IMF, this is separated from the remaining data and is designated as $c_{i}$. The result is a component $r_{n}$. Otherwise, Step (1) is repeated with $h_{i}$ as the input signal.

• If $r_{n}$ is a residual, the original dataset x is decomposed into n IMFs $c_{i}$ and a residual $r_{n}$, which can be either the mean trend or a constant. Otherwise, Steps (1) and (2) are repeated.

• The EDM procedure can be represented as the following equation

  $$\begin{aligned} x = \sum _{i=1}^{n}c_{i} + r_{n} \end{aligned}$$

  (3)

Fig. 3.

Empirical Mode Decomposition with 3 levels.

Continuous Wavelet: It is an application of the wavelet transform that employs arbitrary scales and nearly arbitrary wavelets [19]. Wavelets are brief oscillations localized in time and frequency. In this transformation, the signal is decomposed into a collection of wavelets through the wavelet method, corresponding to a distinct frequency band. The Continuous Wavelet Transform (CWT) generates a time-frequency representation of a signal with best location found by the model in both the time and frequency dimensions.

The CWT generates a graphical representation called a spectrogram that shows the intensity of different frequencies along time. Each point in the spectrogram reflects the intensity of the associated frequency component at a specific time. CWT is especially useful for analyzing dynamic signals, where frequency characteristics can vary over time, and the choice of wavelet and scale parameters is critical in CWT, as it determines the sensitivity of the transform to various signal characteristics. CWT function of an input signal x(t) can be represented as

$$\begin{aligned} W(a,b) = \frac{1}{\sqrt{a}}\int _{-\infty }^{\infty }x(t)\psi ^{*}(\frac{t - b}{a})dt \end{aligned}$$

(4)

where a controls the length of the wavelet, b controls the position of the wavelet along the time axis, and $\psi ^{*}(\frac{t - b}{a})$ is the complex conjugate version of the wavelet $\psi (\frac{t - b}{a})$.

4.3 Common Spatial Pattern

Common Spatial Pattern (CSP) is a technique for spatial filtering to extract features from signals. CSP aims to find a set of spatial filters that maximize the difference between signals from two classes based on their covariance matrices. Its application is given by

$$\begin{aligned} Z_{i}=W^{\text {T}}\times E_{i} \end{aligned}$$

(5)

where $Z_{i}\in \mathbb {R}^{S\times T_{s}}$ is the matrix obtained when we multiply the i-th trial $E_{i}\in \mathbb {R} ^{S\times T_{s}}$ with the transformation matrix $W^{T}\in \mathbb { R}^{S\times S}$. S and T represent the number of electrodes and the size of the signal collected per electrode in each trial. The correlation matrices of each label are used to find the transformation matrix W, and it is defined by

$$\begin{aligned} {\varSigma }^{(c)} = \frac{1}{N_c}\sum ^{N_c}_{n}E^{(c)}_{n}{E^{(c)}_{n}}^T \end{aligned}$$

(6)

where ${\varSigma }^{(c)}\in \mathbb {R}^{S\times S}$ is the correlation matrix between the electrodes, $N_c$ is the number of training samples of class C, and $E^{(c)}_{n}$ is the n-th training sample of class C. Maximization is restricted so that the sum of the diagonal matrices formed by the eigenvalues is equal to the identity matrix, that is, ${\varLambda }^{(1)} + {\varLambda }^{(2)} = I$ This problem is equivalent to solving the generalized eigenvalue problem, defined as

$$\begin{aligned} {{\varSigma }^{(1)}}W = ({{\varSigma }^{(1)}}+{{\varSigma }^{(2)}})W\varLambda \end{aligned}$$

(7)

Finally, the following equation is applied to Z to extract its features:

$$\begin{aligned} f_{i}=log\Bigg (\frac{diag(Z_{i}Z_{i}^T)}{tr(Z_{i}Z_{i}^T)}\Bigg ) \end{aligned}$$

(8)

where $f_{i}$ represents the feature vector of $Z_i$, and $diag(\cdot )$ and $tr(\cdot )$ represent the main diagonal and trace of the matrix, respectively. The number of CSP filters is equal to the number of electrodes. However, only the first and last filters are used. As CSP maximizes the variance of class 1 in the first eigenvectors and class 2 in the last, a feature vector $v_i$ is created where its elements are the m/2 first components and m/2 final components of $f_i$.

4.4 Proposed Multi Band Pipelines with Euclidean Alignment

A temporal filter improves the signal-to-noise ratio and, when applied parallel to the data, can create a new set of filtered data. In this work, we propose these filters with EA to evaluate their combinations. CSP is adopted as a base model, as it is a standard classification approach for MI [7]. We propose here two ways to use EA with multi-band filtering: using the EA before and after the multi-band filter. Also, one can observe that many multi-band filters can be used. This work focuses on the temporal filter models presented in Sect. 4.2. The pipelines described in Fig. 4 are in order: the baseline model with EA [7], the use of a temporal filter before the EA step, and the application of EA before the temporal filter. As shown in Fig. 4, we used LDA as the classifier [14, 15].

Fig. 4.

Literature and proposed architectures using Euclidean Alignment.

5 Computational Experiments

We performed the experiments to evaluate the proposals in Sect. 4. A 5-fold stratified cross-validation was performed to evaluate the proposed approaches. The non-parametric Kruskall-Wallis statistical test was used as a multi-group test followed by Dunn’s test as post-hoc with the Bonferroni correction. The codes implemented for this work are publicly available¹.

5.1 Datasets

The PhysionetMI dataset [6] was used to perform the experiments. That dataset has the recorded data of 109 subjects with 14 sessions per subject. The acquisition was performed using an EEG with a sample rate of 160 Hz and 64 monopolar electrodes based on the 10–10 electrode system [17]. For the first two sessions, a baseline with eyes open and closed was recorded without any task to execute. For other sessions, there are two classes per session, which is sessions: (i) Left-fist and right-fist movement: 3, 7, 11; (ii) Left-fist and right-fist motor imagery: 4, 8, 12; (iii) Both-fists and both-feet movement: 5, 9, 13; and (iv) Both-fists and both-feet motor imagery: 6, 10, 14. Each session with a task has 15 trials, and each trial has 8 s. The cue for the task is presented for 1 s, followed by 3 s of the task execution. Moreover, there is a 4-second break between each trial. The steps of each trial are shown in Fig. 5. Due to the binary classification nature of the models presented here, we only used Left-fist and right-fist motor imagery in our experiments.

Fig. 5.

Timing scheme of a trial from the PhysionetMI dataset.

5.2 Parameters Used

The computational experiments were performed with two subsets of electrode positions: (i) 3 electrodes: C3, Cz, C4; and (ii) 8 electrodes: FC3, C3, CP3, FCz, CPz, FC4, C4, CP4. We reduced the number of electrodes in those 2 cases, as it is proven that a smaller number of electrodes makes the treatment more comfortable for the patient, faster to prepare the treatment, and less expensive [11, 20]. In the future, we will be able to use this electrode parameter to acquire data from more individuals, without losing model accuracy. The positions of the electrodes above the brain motor area are: (i) above the primary motor area when using 3 electrodes, and (ii) the supplementary motor and premotor areas in addition to those in (i) when using 8 electrodes [8].

Furthermore, we created two new classes: left-fist and right-fist, by combining the movement and motor imagery of each one. Only those 2 new classes were used as it is the most common approach for post-stroke rehabilitation BCIs [20]. All acquired data was down-sampled to 128 Hz using a cubic spline. When analyzing the dataset, the EEG trials of subjects 88, 94, and 101 were unavailable, resulting in these subjects being removed from our computational experiments. Considering this, the experiments were performed using data from the remaining 106 subjects. The data window for training started 0.5 s after the cue and lasted for 2 s. The same window bounds were used for the testing phase.

We created 11 models to evaluate our proposal. The first models serve as baselines, namely: CSP, EA-CSP, and $(\cdot )$-CSP, where $(\cdot ) \in$ {FB, EMD, CWT}. After that, the models with EA applied after the multi-band have the form $(\cdot )$-EA-CSP. Finally, we have the models where EA is applied before the multi-band in the form EA-$(\cdot )$-CSP. We performed preliminary experiments, and after that, we set the hyperparameters of the models as: The bandpass was implemented using the Fast Fourier Transform (FFT), between 4 and 40 Hz. For CSP, we used $m=4$. The filter bank used nine bandpass filters: (4, 8), (8, 12), (12, 16), (16, 20), (20, 24), (24, 28), (28, 32), (32, 36), (36, 40). EMD used the smallest amount of IMF less 1, with nine being the maximum limit of this value, and CWT was implemented with five levels. For each target subject, the source data for EA had all other subjects except the target, and the solver for LDA was SVD.

5.3 Results Using 3 Electrodes

The experiments were performed with the settings presented in Session 5.2. Table 1 presents the results for the three-electrode case, where the accuracy for EA-CSP and MB-CSP was worse than EA-MB-CSP and MB-EA-CSP for all MB methods. Filter Bank was the best temporal filter method regardless of the use of EA. EA-FB-CSP outperformed the remaining approaches. The statistical analysis presented a significant difference p-value $<0.001$ for all methods when compared to EA-FB-CSP. Furthermore, FB-EA-CSP was the second-best model, with a statistical difference concerning the remaining models. These results pointed out that Filter Bank is the best temporal filtering evaluated here to be used with EA, regardless the order of its application in the BCI pipeline. It is also possible to note that EA before multi-band reached better results in all tested cases.

Table 1. Accuracy for the scenario with 3 electrodes.

5.4 Results Using 8 Electrodes

For the 8 electrodes case, and their results are presented in Table 2. The results are similar to those observed in the 3 electrodes case. EA-FB-CSP outperformed the remaining approaches. The use of EA before multi-band performed better than that applying EA after it, and Filter Bank is the temporal filter model with the best result. Such as in the 3-electrode case, the p-value for all methods compared to EA-FB-CSP was lower than 0.001. When comparing both electrode settings, 3 electrodes reached better results than 8 electrodes for all cases. That difference between 3 and 8 electrodes was statistically significant for all models. In the same way as the 3-electrode case, the EA-MB-CSP pipeline was more robust than the other two pipeline models. The results pointed out that EA as the first step of the BCI pipeline makes the model more robust between subjects.

Table 2. Accuracy for the scenario with 8 electrodes.

5.5 Performance Profile Analysis

Fig. 6.

Performance profiles of the results. Each problem is a combination of subject and number of electrodes.

The performance profiles (PPs) were used to conduct an overall analysis concerning the accuracy value [20], where each pair (subject, electrode case) is a problem. PPs present how many problems were solved with less than $\tau$ times the best value for each problem. Figure 6 contains PPs regarding the results of the 11 methods evaluated in this work. The following conclusions can be highlighted: (i) EA-FB-CSP was the model with the best result for most subjects, since it had the largest value for $\tau = 1$ (about 90% of the 212 problems). (ii) EA-FB-CSP obtained the best overall performance in the average case, as this approach achieved the biggest area under the PP curve; and (iii) EA-FB-CSP is the most robust method, with at more than 10% worse than the best case. Other results in PPs are: (i) FB-EA-CSP was the second method for the average result but the 4-th most robust method; (ii) the three most robust cases used EA before the multi-band; and (iii) EA-CSP was better and more robust than all cases which did not use EA.

6 Discussion

All models in the 3-electrode case obtained better results than their counterparts in the 8-electrode case. One can observe that the amount of electrode raises the dimensionality for LDA with no relevant information, leading to worse results. LDA performs better when the number of trials per area in the search space is high enough to fill it. When the dimension increases, the search space gets bigger and less dense. That explains that models in the 3-electrode case have better results than the 8-electrode case.

The standard BCI pipeline is formed by the following steps: Temporal Filtering, Spatial Filtering, Feature Extraction, Feature Selection, and Classifier. EA can be represented as a spatial filter due to its linear transformation on the input signal. Therefore, better results are expected when EA is used after the temporal filtering, such as in the BCI pipeline. However, the results found in the experiments demonstrate the opposite, leading to questions regarding the BCI stage’s order. Spatial filters before the temporal step can result in new configurations for the BCI pipeline. Filter Bank was the best temporal filter tested here. This result is expected, as it takes the signal in frequency sub-bands, being able to select separately $\mu$ (8–13) and $\beta$ (13–30) rhythms, which is crucial for motor imagery activities [23]. The other temporal filtering techniques applied in this work do not use the signal spectrum as effectively, which explains their results in comparison with Filter Bank.

7 Conclusion

BCI-based MI models can be used for post-stroke rehabilitation. There are some difficulties despite that, such as long calibration periods during the training phase and low resolution of the signal obtained. We proposed Euclidean Alignment (EA) with multi-band temporal filters to reduce the impact of these two conditions. The models were created using Filter Bank, Empirical Mode Decomposition, and Continuous Wavelet as temporal filter techniques. Each model uses one of these temporal filters, either before or after EA. We evaluated the models in 2 cases: with 3 and 8 electrodes in the PhysionetMI dataset. The models used in this work demonstrate how EA and multi-band temporal filters can be used together to acquire more robust models. They also demonstrate that changing the steps’ order can improve the quality of the solutions. The best model in the computational experiments performed here is EA-FB-CSP-LDA, with an improvement of approximately 44% for three electrodes and 43% for eight electrodes when compared to EA-CSP-LDA, reaching an accuracy of 80% and 73.33%, respectively. EA before the temporal filtering presented results better than those obtained by models with EA after the temporal filtering.

New experiments using Feature Selection or Subject Selection can be performed to improve those results. Our results also pointed out that Spatial Filtering before Temporal Filtering has prominent potential for new BCI pipelines.