key: cord-027732-8i8bwlh8 authors: Boudaya, Amal; Bouaziz, Bassem; Chaabene, Siwar; Chaari, Lotfi; Ammar, Achraf; Hökelmann, Anita title: EEG-Based Hypo-vigilance Detection Using Convolutional Neural Network date: 2020-05-31 journal: The Impact of Digital Technologies on Public Health in Developed and Developing Countries DOI: 10.1007/978-3-030-51517-1_6 sha: doc_id: 27732 cord_uid: 8i8bwlh8 Hypo-vigilance detection is becoming an important active research areas in the biomedical signal processing field. For this purpose, electroencephalogram (EEG) is one of the most common modalities in drowsiness and awakeness detection. In this context, we propose a new EEG classification method for detecting fatigue state. Our method makes use of a and awakeness detection. In this context, we propose a new EEG classification method for detecting fatigue state. Our method makes use of a Convolutional Neural Network (CNN) architecture. We define an experimental protocol using the Emotiv EPOC+ headset. After that, we evaluate our proposed method on a recorded and annotated dataset. The reported results demonstrate high detection accuracy (93%) and indicate that the proposed method is an efficient alternative for hypo-vigilance detection as compared with other methods. Hypo-vigilance has been one of the major causes of accidents in many areas such as driving [1] , aviation [2] and military sector [3] . Hence, the drowsiness problem has gained great interest from researchers. This is today a real up to date problem within the current Covid-19 [4] pandemic where medical stuff is generally overbooked. In fact, the drowsy condition is expressed predominantly by the emergence of various behavioral signs such as heaviness in terms of reaction, reflex reduction, occurrences of yawning, heaviness of the eyelids and/or the difficulty of keeping the head in the frontal position relative to the field of vision. Many studies [5] [6] [7] [8] have been proposed to detect hypo-vigilance based on biomedical signals such as electroencephalogram (EEG), electrocardiogram (ECG), electromyogram (EMG), and electrooculogram (EOG). Given, its high temporal resolution, portability and reasonable cost, the present work focus on hypo-vigilance detection by analyzing EEG signal of various brain's functionalities using fourteen electrodes placed on the participant's scalp. On the other hand, deep learning networks offer great potential for biomedical signals analysis through the simplification of raw input signals (i.e., through various steps including feature extraction, denoising and feature selection) and the improvement of the classification results. In this paper, we focus on the EEG signal study recorded by fourteen electrodes for hypo-vigilance detection by analyzing the various functionalities of the brain from the electrodes placed on the participant's scalp. Various deep learning architectures [9] exist such as Convolutional Neural Network (CNN), Recurrent CNN (R-CNN), Auto-Encoder (AE), Deep Belief Network (DBN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU). As in [10] , the CNN architecture is the most used to biomedical signals analysis providing a high classification accuracy. Previous related work [11] proposes a hypo-vigilance detection method using CNN by facial features. This method showed a classification accuracy of 92.33%. Likewise [12] , introduces an adaptive conditional representation learning system for driver drowsiness detection based on a 3D-CNN. The proposed system consists of four steps (spatio-temporal representation, data preprocessing, features combination and somnolence detection). The experimental results show a detection accuracy equal to 92.04%. In this paper, we propose a CNN hypo-vigilance detection method using EEG data in order to classify drowsiness and awakeness states. Accordingly, the proposed approach including used equipment are presented in Sect. 2. Section 3 describes the experimental results and the evaluation of the employed method. Finally, a conclusion and future work are drawn in Sect. 4. As shown in Fig. 1 , the realization of the proposed approach is suggested by two primary procedures: data acquisition and data analysis. The following subsections provide a detailed explanation of each procedure. The EEG data acquisition procedure is made up of two main steps which are data collection and data preprocessing. To collect the raw EEG data from participants, we use an Emotiv EPOC+ headset as shown in Fig. 2 [a] for the data acquisition process. The key feature of this headset is a non-invasive Brain computer Interface (BCI) tool designed for the development of human brain and contextual research [13] . The Emotiv EPOC + helmet contains fourteen active electrodes with two reference electrodes (DRL and CMS), as shown in Fig. 2 [b]. The electrodes are placed around the participant's head in the structures of the following zones: frontal and anterior parietal (AF3, AF4, F3, F4, F7, F8, FC5, FC6), temporal (T7, T8) and occipital-parietal (O1, O2, P7, P8). The specific preprocessing steps of the data revolve around the following points which are data preparation, data annotation and data augmentation. During data acquisition, our raw EEG signals may be influenced by various sources of artifacts and noise such as endogenous electrical properties, specific fabrics physical structure, dipolar size variation, muscle shifts and Blinks. Hence, data processing is a preliminary step to denoising the raw signals. We suggest using an infinite impulse response (IIR) filter that manages an impulsive signal within time and frequency domains. Other sophisticated denoising approaches could be considered at the expense of higher computational complexity [14, 15] . - To evaluate each individual's state of exhaustion, we concentrate on the brain areas that are responsible for hypo-vigilance detection. In this regard, different brain waves are targeted such as [16] : • Delta waves refer to consciousness, sleep or deep sleep states. These waves were found in the temporal and occipital conditions with low frequency (less than 4 Hz) and high amplitude. • Theta waves design the relaxation and hypnosis states with a range of frequency between 4 and 8 Hz. Theta waves are extracted from the temporal zone and are produced during the first phase of slow sleep or in deep relaxation state. • Alpha waves refer to waking but relaxed states. These waves are captured in the posterior part, precisely the occipital region, with a frequency interval between 8 and 12 Hz and a low amplitude interval between 20 and 60 µV. • Beta waves relate to alertness states. These waves are captured from the temporal and occipital lobes of the brain. They are characterized by high frequency interval of 12 to 30 Hz with a low amplitude interval of 10 to 30 µV. • Gamma waves refer to hypervigilance states with a frequency interval between 30 to 80 Hz. In the data annotation step, we only use the O1 and O2 electrodes of occipital zone which are responsible for drowsiness sensation. As an annotation example, Fig. 3 indicates the amplitudes of the alpha and theta signals from the two O1 and O2 electrodes reported for a participant in three periods of the day. The relaxation state has been indicated by alpha waves which have a frequency interval between 8 to 12 Hz and an amplitude interval between 20 to 60 µV. The somnolence state has been indicated by theta waves which have a frequency interval between 4 to 8 Hz and an amplitude interval between 50 and 75 µV. In order to reduce overfitting and increase testing accuracy, we use the data augmentation technique [17] which consists of increasing the training set by label-retaining data transformations. The purpose procedure is to extend the data by doubling the vectors from (5850, 2) to (59053, 2) where 5850 (resp. 59053) represents the vector size and 2 represents the class number. The diagram of the neural network simple CNN used in our EEG drowsiness detection approach is represented in Fig. 4 . The proposed simple CNN model is composed of the following six main layers: -The convolutional layers allow the filter application and the features extraction characteristics of the input signals. -The sample-based discretization max-pooling-1D blocks is used to sub-sample each input layer by reducing its dimensionality using a decrease in the number of the parameters to learn, there by reducing calculation costs. Our protocol revolves around the following axes: eight volunteers in which four women and four men aged twenty six and fifty eight with normal mental health. For each participant, we make three recordings of sixteen minutes divided over three day periods (morning, afternoon and evening). To fully understand the condition of the participants, we split the signal into windows to accurately identify these different states. In the proposed simple CNN architecture for EEG signals classification, we use the Keras deep learning library. The different parameters as filters, kernelsize, padding, kernel-initializer, and activation of the four convolutional layers have the same values respectively 512, 32, same, normal and relu. The parameter values of the remaining layers are detailed in the following: -the dropout layer value equal to 0.2 (respect. 0.5) is used to inactivate 20% (respect. 50%) of neurons in order to prevent overfitting. -the Max-Pooling 1D layer is used with a filter size of 128. -The muti-dimensional data output flatting using 1D flatten layer. -For better classification results, two dropout layers are used. The first hidden layer takes a value of 128 neurons. Since a binary classification problem, the second layer takes a value of 1. The choice of the optimization algorithm makes the difference between good results in minutes, hours or even days. There are various optimizers like Adam [18] , SGD [19] and RMS pop optimizer [20] . In our model, we use the SGD optimizer which is more popular [21] . The method of this optimizer is simple and effective for finding optimal values in a neural network. Table 1 presents the hyperparameters choice of our model. For selecting the best accuracy rate of the proposed method, we propose to compare different results recorded by different numbers of electrodes. In [22, 23] , the authors discover that the prefrontal and occipital cortex are the most important channels to better diagnose the hypo-vigilance state. In this regard, we choose the following recorded data: -Recorded data by 2 electrodes (O1 and O2) electrodes from the occipital area. -Recorded data by 4 electrodes (T7, T8, O1 and O2) from temporal and occipital areas. -Recorded data by 7 electrodes (AF3, F7, F3, T7, O2, P8, F8) from prefrontal and occipital areas. -Recorded data by 14 electrodes. For the distribution of our data, we choose 70% for the train part and 30% for the test. Table 2 presents the reported testing and training accuracy respectively with two, four, seven and fourteen electrodes. After convergence the optimum number of test epochs for all the different electrodes results establish a value equal to 80. The best results are given by the recording of 2 electrodes from the occipital area. The curves of testing and training results for recorded data by O1 and O2 electrodes are represented in Fig. 5 . According to results obtained in Fig. 5 , we note that the test accuracy increases after a certain number of epochs and the test loss decreases. To test our system's efficiency we measured the precision, recall and F1-score. Table 3 shows these different measures in our experimental configuration. For comparison purposes, we compare the proposed method with recent drowsiness methodology [24] where the authors propose a driver hypovigilance detection using the Emotiv EPOC+ helmet. The Common Spatial Pattern (CSP) algorithm is used for optimization accuracy of Extreme Learning Machine (ELM). The reported values in Table 4 indicate that our method gives the optimum accuracy value classification. The present work proposes a CNN based approach for Hypo-vigilance detection. In order to create a EEG dataset, we recorded raw EEG data using Epoc+ headset. The suggested system achieves an average classification accuracy to 93.94% by testing it on a real dataset of eight participants. In future work, we will focus to improve classification accuracy with large datasets. Additionally, fusion with other biomedical signals should be also considered to improve the classification accuracy. Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. Noise robustness analysis of performance for EEG-based driver fatigue detection using different entropy feature sets Fatigue detection in commercial flight operations: results using physiological measures Simulated sustained flight operations and performance, part 1: effects of fatigue Covid-19 pandemic by the "real-time" monitoring: the Tunisian case and lessons for global epidemics in the context of 3PM strategies Electromyogram signal based hypovigilance detection Real-time ECG-based detection of fatigue driving using sample entropy Exploring neuro-physiological correlates of drivers' mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and fNIRS data Muscle fatigue detections during arm movement using EMG signal A state-of-the-art survey on deep learning theory and architectures 1-D convolutional neural networks for signal processing applications Drowsy driver detection using representation learning Driver drowsiness detection using conditionadaptive representation learning framework Analysis of performance metrics using Emotiv EPOC+ Hybrid sparse regularization for magnetic resonance spectroscopy Sparse signal recovery using a Bernouilli generalized gaussian prior Analysis of the meditation brainwave from consumer EEG device A novel deep learning approach with data augmentation to classify motor imagery signals Deep learning for EEG data analytics: a survey Two classes classification using different optimizers in convolutional neural network Automatic microemboli characterization using convolutional neural networks and radio frequency signals Optimization of deep learning using various optimizers, loss functions and dropout Real time fatigue-driver detection from electroencephalography using Emotiv EPOC+ Drowsiness analysis using common spatial pattern and extreme learning machine based on electroencephalogram signal