key: cord-0641505-ulnfzh1s
authors: Rostaminia, Soha; Homayounfar, S. Zohreh; Kiaghadi, Ali; Andrew, Trisha L.; Ganesan, Deepak
title: PhyMask: Robust Sensing of Brain Activity and Physiological Signals During Sleep with an All-textile Eye Mask
date: 2021-06-13
journal: nan
DOI: nan
sha: 9d63bab46bacdf3deb79285c4d07d95bc1708e4c
doc_id: 641505
cord_uid: ulnfzh1s

Clinical-grade wearable sleep monitoring is a challenging problem since it requires concurrently monitoring brain activity, eye movement, muscle activity, cardio-respiratory features and gross body movements. This requires multiple sensors to be worn at different locations as well as uncomfortable adhesives and discrete electronic components to be placed on the head. As a result, existing wearables either compromise comfort or compromise accuracy in tracking sleep variables. We propose PhyMask, an all-textile sleep monitoring solution that is practical and comfortable for continuous use and that acquires all signals of interest to sleep solely using comfortable textile sensors placed on the head. We show that PhyMask can be used to accurately measure sleep stages and advanced sleep markers such as spindles and k-complexes robustly in the real-world setting. We validate PhyMask against polysomnography and show that it significantly outperforms two commercially-available sleep tracking wearables, Fitbit and Oura Ring.

But achieving high-quality clinical-grade sleep monitoring with a comfortable wearable device is complicated by the number of sensor modalities and locations that need to be simultaneously monitored. Sleep is a complex process and monitoring it accurately necessitates many sensors placed at different places on the body. This includes sensors to measure brain activity, eye movement, muscle activity, as well as cardio-respiratory features and gross body movements. To monitor all of these variables, clinical-grade sleep monitoring systems (also referred to as polysomnography or PSG) place electrodes on the head as well as several other locations on the body. However, this is impractical for daily use.

Sleep sensing devices in the market therefore have to tradeoff between comfort and accuracy while monitoring sleep parameters (see Figure 1) . A key challenge is that measuring brain electrical activity (Electroencephalography, EEG) is both the most onerous aspect of sleep sensing (since it requires electrodes on the head) and the most valuable sensing modality (since sleep is best measured via brain activity). Thus, sleep sensing devices can be divided into two groups -those that favor comfort over accuracy and rely on surrogate measures of sleep such as heart rate, breathing, and body movement signals rather than EEG, and those that favor accuracy over comfort and retain the use of electrodes to measure the electrical activity of the brain and eye movements induced by sleep but try to do so without compromising too much comfort.

Measuring surrogate signals: The first group, i.e. sleep monitors that use surrogate measures of sleep, includes contactless sleep monitors like instrumented bedding and bedside sleep monitors [50, 57, 61, 72] as well as wearable devices such as the Fitbit and Apple Watch [13, 19] .

While these are comfortable, they sacrifice fidelity and precision. Since the surrogate signals only capture a coarse temporal structure of sleep, they provide only coarse-level sleep metrics like sleep quality and, in some cases, macro-structural analysis of sleep such as sleep stages. In addition, metrics provided by these devices

In this work, we present a complete textile-based head-worn sleep sensing system that bridges the gap in existing sleep measurement devices in two ways. First, we show that a single head-worn device that leverages solely textile-based sensors can provide nearly all the parameters that are used in clinical-grade sleep monitoring without requiring rigid sensing elements and thereby without sacrificing comfort. Second, we show that such a device can provide accurate estimates of both macro and micro-structures of sleep including sleep stages, spindles and K-complexes.

Our work builds on our previous efforts on developing individual sensors that provide part of the solution. In the previous work [45] , we have introduced two sensors that we leverage in this work: a) a new thread-based, reusable wet electrode that achieves the high signal quality of commercial wet electrodes as well as the comfort and unobtrusiveness of dry electrodes (our preliminary results showed that these electrodes were viable for measuring eye movement patterns), and b) a fabric-based piezoionic pressure sensor [44] that is sensitive to ballistic signals from heartbeats.

In this work, we build on the above to provide a holistic multi-modal solution that can provide reliable and robust measures of sleep in a natural setting. Our work makes notable contributions towards a practical daily-use system that is comfortable yet accurate. From a device perspective, we show for the first time that a) a fabric-based electrode can be used to accurately measure EEG signals with high signal-to-noise, and that b) textile-based pressure sensors on the sleep mask can allow us to sense tiny head movements induced by heartbeats and respiration in various sleep postures. From an analytic perspective, we describe a signal processing and machine learning pipeline that allows us to extract both high-level physiological features such as heart rate and breathing rate, as well as micro-events of brain activity during sleep such as spindles and k-complexes. Put together, PhyMask is a comfortable textile-based sensing platform that can simultaneously sense brain activity (EEG), eye movement (electrooculography, EOG), physiological parameters (respiratory and cardiac rhythm), gross body movement, and sleep posture from the head and infer both macro-and micro-structural parameters of sleep.

We design and fabricate a fully functioning prototype of a comfortable all-textile sleep tracking device (shown in Figure 2 ) that includes the fabric-based sensing elements, fabric-based wires, low-power circuit for signal amplification and data acquisition, a wireless radio for data transfer, and tailored algorithms for physiological signals and sleep micro-events tracking.

We perform exhaustive data collection studies to validate our device and compare it against polysomnography ground truth and two popular sleep trackers, Fitbit wristband and Oura Ring, for continuous long-term sleep tracking.

Our results show that:

• We have high accuracy for measuring heart rate (median error of 1.7 beats/min) and respiration (median error of 1 resp/min), and can easily capture gross motor activity including posture changes. • The PhyMask can measure sleep stages almost as well as polysomnography ground truth with an F1 score of 0.91, and greatly outperforms Fitbit (F1 score of 0.64) and Oura Ring (F1 score of 0.7). • The PhyMask can be used to detect advanced sleep markers such as spindles and K-complexes that are typically not provided by commercial wearable devices. We show that PhyMask detects spindles and K-complexes with accuracy of 85% and 86%, respectively.

In this section, we first look at the key sleep markers that we need to measure in order to provide a reliable sleep tracking solution. We then look at the existing sleep tracking solutions with respect to two questions: a) which of these sleep markers can these solutions provide? and b) how comfortable and unobtrusive are these techniques? We explain the PhyMask solution and how it overcomes the accuracy and aesthetic shortcomings of current state-of-the-art sleep tracking devices. However, sleep is also rich in shorter timescale phasic EEG events such as K-complexes, spindles, delta wave bursts, and others [63, 86] . While macrostructural variables have received more attention in past research, recent work has paid more attention to microstructural variables due to important clinical correlations to age-related sleep changes and other age-related disorders like Parkinsons and Alzheimers [84] . Many age-related sleep changes are not captured by traditional sleep stage scoring -for example, research has shown that agingrelated changes were reflected particularly in fast spindle density, K-complex density, and delta power during N3 sleep-than on conventional sleep staging variables [81] .

Biosiognal characteristics during sleep: A number of biosignals need to be measured to fully characterize micro and macro-structural parameters of sleep. Table 1 summarizes the biosignals characteristics during each sleep stage i.e. Rapid Eye Movement (REM), non-REM N1, non-REM N2, and non-REM N3. Each sleep stage is characterized by brain waves of specific frequencies and/or amplitudes and is also associated with certain types of eye movements and muscle activities. Therefore, brain activity (EEG), eye movement (EOG) and muscle activity (EMG) measurements are required for accurate sleep staging. Brain activity, in particular, is considered the most important signal for high-quality sleep stage monitoring [75] .

Laboratory-based polysomnography (PSG), which is considered the most clinically accurate method of monitoring sleep also includes monitoring of several other parameters which are relevant to holistic understanding of sleep. Cardio-respiratory parameters are particularly useful for studying and analysis of sleep disorders such as apnea. Thus, measurements of respiration and heart features are also included in laboratory-based polysomnography (PSG). In addition, gross body movements during sleep is also important when studying sleep disorders such as periodic leg movements.

Sleep tracking devices typically capture a subset of the signals in Table 1 due to limitations of comfort, sensor position, and other factors. We look at how well different types of sleep monitoring solutions can monitor these parameters.

Non-wearable sleep trackers. There has been a number of non-wearable sleep tracking approaches that attempt to monitor sleep indirectly by instrumenting the environment around the user. For example, Jia et al. leverage highly sensitive geophones and embed it into bedding to measure the seismic motions induced by individual heart beats and slow moving signals from respiration [50] . Commercial MEMS accelerometer-based and piezoelectric-based units are also available, that can measure body movement and heart rate based on ballistocardiography signals measured via the bed during sleep [6, 11] . Several approaches also use vision-based and depth camera-based methods to find physiological variables such as respiration [28, 64] , heart rate [57] , pulse oximetry [88] , head posture [37] , and body movement [68, 91] during sleep. Another body of work is on radar-based sensing of respiration and heart rhythm [23, 67, 72] . These methods use FMCW or UWB radars and measure changes in the displacement and the doppler shifts due to respiration and ballistics of the heart.

While non-contact sleep tracking solutions have the advantage of being unobtrusive and comfortable, they tend to be imprecise and noisy. The imprecision stems from the inability to observe the most valuable signal for sleep monitoring i.e. EEG, and relying on other parameters such as body movements (chest movement during respiration and gross body movements) to infer sleep markers. Noisy measurements results from the fact that the signal is confounded by proximate activity such as multiple individuals on the bed [49, 62] or other moving objects in the vicinity like a fan.

Wrist-worn wearable sleep trackers. There are many wearable devices in the market for sleep sensing, most of which use hard electronic components such as photoplethysmography (PPG) and IMU to measure the pulse wave and body movement on the wrist or fingers (e.g. Fitbit [13], Polar Vivofit [14], Actiwatch [10] , Whoop [21] , and Oura Ring [17] ).

The main disadvantage of these solutions is the inability to monitor the EEG and EOG signals, which is considered critical for high-quality sleep analysis. Hence, they solely rely on cardiac, respiratory, and gross body movement information for sleep stage tracking which results in poor performance. Several studies in research settings have reported on the validity of the wearable sleep trackers compared to polysomnography (PSG). For example, Moreno-Pino et al. validate two models of Fitbit sleep trackers (Charge 2 and Alta HR) against PSG and find statistically significant differences between PSG and Fitbit measures for all sleep stages except for REM sleep [66] . Also, Miller et al. examine Whoop against PSG and report the sensitivity to light sleep, deep sleep, REM, and wake, 62%, 68%, 70%, and 51% respectively [65] .

Head-worn wearable sleep trackers. Another category of sleep tracking devices is headworn wearables. Headworn solutions are more promising as they allow us to obtain biopotential signals (EEG and EOG) from the head. Kim et al. leverage gel-based standard adhesive electrodes on a headband device and collect EEG, EOG, and EMG signals for sleep tracking purpose [55] . Phillips Smartsleep [18] also use sticky disposable electrodes placing behind the ear and obtain EEG signal during sleep. Despite the high signal quality of these electrodes, they are uncomfortable due to the adhesive and are not practical for long-term wear, since once the gel dehydrates, the electrode loses its functionality and should be replaced. Therefore, dry electrodes are often preferred in wearable devices. Kuo et al. use graphene-based electrodes in a sleep mask, for tracking eye movement signal during sleep [58] . Furthermore, the commercially available computational headbands, Muse [15] , Dreem [12], Brainbit [20], and Neuroon [7] , use dry electrodes for obtaining EEG on the forehead. They also have a PPG sensor for cardiac and respiration rate tracking, and other sensors.

The main drawback of these systems is the rigid structure of the device and the use of hard sensing components that touch the skin on the sensitive head areas. These can make the system highly uncomfortable in different sleep postures and not ideal for continuous long-term sleep tracking. Another debatable aspect of the current head-worn solutions is the quality of the acquired biopotential signals. Majority of these technologies use dry electrodes to avoid adhesives and increase the comfort factor, however, this results in considerable increase in motion artifact [76] .

PhyMask. While most of the current solutions focus on collecting a few of the essential signals needed for precise sleep monitoring, PhyMask captures all these signals i.e. EEG, EOG, and heart and respiratory rate, while providing information on head posture and body movements.

This allows our device to extract information about sleep macrostructure as well as microstructure in contrast to existing devices which only provide macrostructural parameters. To the best of our knowledge, PhyMask is the first solution that can also accurately detect sleep micro-events, i.e. spindle and K-complex in addition to macrostructural parameters.

Since PhyMask relies on textile-based sensing elements, we look at the state-of-the-art textile-based sensing solutions for monitoring these parameters.

Physiological and physical signals sensing. Physiological and physical sensing through fabric-based elements has gained a lot of interest during the past decades. Some of the existing methods and requirements of smart textiles are surveyed in [26] . Many of these devices are based on conductive fabric electrodes embedded into tight-fitting clothing. There has been some work on measuring impedance changes for physiological measurements -for example, [70] integrates piezoelectric elements in a garment and acquire electrocardiogram, respiration, and activity information. In our previous work, Phyjama [54] , we also showed that we can measure heart rate and respiration as well as sleep posture through resistive pressure and triboelectric patches sewn into a loosely worn sleepwear. To the best of our knowledge PhyMask is the first system that captures the physiological and physical signals (cardiac, respiration, general body movement, and head posture) through fabric-based elements robustly from the head.

Biopotential signals sensing. There has been some studies on design of textile-based dry electrodes. For example, silver-coated [59] or graphene-coated [41, 77] fabrics, or polymeric foams [60] have been embedded as a dry electrode into a headworn to obtain EOG and EEG signals. Pani et al. [69] use PEDOT:PSS-based electrodes to measure ECG signal. While dry electrodes are unobtrusive and comfortable to use, they are highly susceptible to noise. The fundamental challenge in reliably extracting biopotential signals through wearable electrodes is to design an electrode enjoying both the signal quality of traditional wet electrodes and the comfort of the dry ones. Shu et al. [83] and Alba et al. [24] propose semi-wet textile electrode for EEG measurement. However, to the best of our knowledge the existing solutions does not have the longevity, wash-stability, and recoverablity that are required for long-term use of the electrodes.

PhyMask. PhyMask presents a complete system that incorporates several of these textile-based sensor innovations in a single platform to monitor sleep. PhyMask uses the thread-based, reusable wet electrode to measure biopotential signals that has the signal quality of commercial wet electrodes and the comfort and unobtrusiveness of dry electrodes. We also use a pressure-sensitive, ionic fabric electrode to capture pulse wave-forms from the head. In order to further minimize using discrete electronic hard components at sensitive pressured locations, we inter-connect the all-textile patches by using silver-plated nylon threads as wires that are shielded in cotton.

A key design objective of PhyMask is to sense the biosignals relevant to sleep while eliminate all hard sensing components and relying solely on textile-based and soft sensing elements that can be embedded in a sleep mask. While it is possible to obtain sleep markers using a combination of devices, say one on the head for EEG and EOG [15, 18] , and another on the body for cardio-respiratory rhythm and movement [17, 54] , it is more ideal to obtain all metrics from a single device on the head. Since the head area is sensitive, we remove all hard sensing components from the head-worn device and rely solely on textile-based and soft sensing elements that can be embedded in a sleep mask. Such a design can eliminate rigid sensing elements from being in direct contact with the skin on the head area. We note that unlike sensing elements that need to be in contact with the skin, other rigid components like the microcontroller, radio and battery are easier to place in a more conducive location. We now describe the design of such a device and how we optimized it to extract the markers of interest.

In order to extract biopotential signals such as EEG and EOG in a comfortable manner, we need a fabric-based electrode that is comfortable, robust, and provide reliable high signal-to-noise ratio (SNR). PhyMask leverages a recent innovation; a fabric-based, reusable wet electrode that has a layer of composite hydrogel, which when hydrated, mechanically behaves like the foams used in the standard electrodes [45] . This is sufficiently cushion-like to minimize motion artifacts in the absence of any harsh skin adhesives. The advantage of these electrodes is that they overcome the aesthetic drawback of requiring adhesives to have contact with the skin while simultaneously providing strong signal integrity. In addition, they address the lack of reusability of the commercial wet electrodes.

In order to reduce the computational cost of the system, we need to choose the minimum number of electrodes that can be embedded into a sleep mask and provide reliable EEG and EOG signals concurrently. The typical electrode placement for EOG and EEG can be too complex. For example, EOG alone requires five electrodes placed as illustrated in Figure 3a .

To reduce the number of electrodes, we look at a more restricted placement. In order to be able to capture strong EEG signals, the electrodes should be placed closer to the brain area. Thus, we place one electrode on top of each left and right eyes (HG1 and HG2, respectively) and one in the middle (HG0), serving as both reference and common ground (shown in Figure 3b ). HG1 and HG2 electrodes are respectively closest to the Fp1 and Fp2 sites (illustrated in Figure 13 ) according to the standard 10-20 EEG recording guideline [22] and can conveniently capture horizontal EOG signal which is sufficient for the purpose of sleep stage tracking. 

The next question is how to obtain robust physiological signals such as pulse and breathing, as well as gross motor activity on the head using comfortable textile-based sensing elements. Traditional methods for capturing cardiac signals on the head focus on placing sensors at the supraorbital artery position (shown in Figure 3e ). The cardiac signal at this location can be measured with rigid optical sensors to obtain PPG (e.g. the MUSE headband). Furthermore, in the prior work on textile-based sensors, we have looked at using this artery to obtain pulse due to pressure changes (Chesma [45] ). However, the issue is that textile pressure sensors do not work well when the signal is present in a small surface area like a single artery. This leads to a weak and noisy signal that is highly sensitive to small changes in sleep position.

We address this limitation using a new method for measuring cardiac signals in the head area while using textile-based sensors. A key observation that we make is that the cyclical movement of blood from the heart to the head via the abdominal aorta and the carotid arteries causes the head to move imperceptibly in a periodic manner. Similar oscillations can also be observed during respiration since each inhale and exhale action results in body movements [27] . Therefore, unlike the previous work which attempts to sense the artery pulse in a very localized region, we leverage fabric-based pressure sensors across a larger surface area to measure both the pressure exerted by blood pulsing through the facial artery as well as the small subtle head vibrations caused by blood pumping and respiration.

In order to detect these vibrations under various sleep postures, we place the pressure patches on the back (P1), left (P2), and right (P3) of the sleep mask, where at least one of the patches is pressed on the pillow under the user's head weight in different sleep postures (shown in Figure 3b -d). For precise placement of the pressure sensors and in order to achieve the maximum heart rate sensing ability, we look more closely at the physiology of the head. Figure 3e illustrates the anatomy of the arteries passing through the head. The superficial temporal artery is a major artery of the head. It arises from the external carotid artery when it splits into the superficial temporal artery and maxillary artery. Its pulse can be felt above the zygomatic arch, above and in front of the tragus of the ear. Therefore, P2 and P3 sensors are made bigger and placed slightly tilted (∼ 15 • ) on the temple area, allowing for the pulse waveform from the frontal branch of superficial temporal artery to be measured. This layout of the sensors also enables detecting the head posture on the pillow and can capture the pressure changes caused by gross body movement.

The difference between measuring pulse at a single location versus measuring head oscillations over a broader surface area is illustrated in Figures 4a and 4b , where the participant is lying on their right and back, respectively.

We see that the pulse signal acquired with the sensors placed on the supraorbital artery is weak and imprecise, while PhyMask's sensors that are optimized to extract head oscillations can pick up clear pulse signal. Figure 2a shows the integrated PhyMask platform, which incorporates three biopotential electrodes and three pressure sensors. We made two biopotential electrodes with functional area of 1 cm x 1 cm and one electrode with functional area of 1 cm x 2 cm, serving as reference. We made the reference electrode slightly bigger to ensure better skin-electrode contact and hence more stable signal during long wear. The layered structure of the electrode (shown in Figure 2d ) is comprised of an array of conductive silver-plated nylon threads, serving as the charge collector, on top of a hydrophobic backing fabric. This conductive array is then coated with silver chloride that provides the ion conductivity required for transducing signals from the ionic form in the body to the electrons in the wires. By taking advantage of the initiative chemical vapor deposition (iCVD) of poly(hydroxy ethyl acrylate) (pHEA) on a pharmaceutical grade of silver gel, we developed a reusable composite hydrogel on top of the electrode. Finally, an open-weave cotton gauze is placed on top and a cotton flannel framed the hydrogel to protect it from mechanical abrasions. The wide pores of the gauze fabric enable rehydration of the hydrogel and its direct contact with the skin while protecting it from being rubbed away. All the fabrics used in this electrode are coated with poly(perflurodecyl acrylate) using iCVD. The hydrophobic nature of this polymer prevents the fabrics from absorbing water/perspiration rather than the hydrogel.

As it can be seen in Figure 2b , the pressure sensor is comprised of an ion conductive active layer sandwiched between two silver-coated conductive fabrics as electrodes. The active layer is made of a cotton gauze fabric coated with a poly(siloxane) polymer, poly(N-propylsilyl-N,N,N,-trimethylammonium chloride) through a solution-phase functionalization process. Through the application of a compression stress on the sensor, the mobile chloride counterions relocate on the surface of the fabric and the ions movement in addition to the reduction of the air gap between the layers lead to a decrease in the impedance of the pressure sensor. The functionalized ion-conductive layer was further encapsulated with a poly(perfluoroalkylsiloxane) coating through iCVD. The hydrophobic nature of this coating protects the ion-conducting fabric against common aging processes, such as erosion during laundering or air oxidation, ensuring that the ionic conductivity of the sensor will not be washed away or diluted if the pressure patch comes into contact with sweat [45] . We made a large patch of functionalized ion-conductive layer, and then cut it into four 12 cm x 4 cm and two 6 cm x 4 cm sheets, each of which was sewn around the perimeter onto a sheet of silver fabric. Sewing together each pair of these joined gauze-silver sheets yielded three resistive sensors with a 4-layer structure.

All of these fabric electrodes and sensors are tightly sewn onto a commercially-available sleep mask -purchased from Amazon 4.5/5 with more than 4000 reviews, it was chosen because of its lightweight and adaptability to a wide variety of head shapes and sizes. For maximum comfort and to minimize the number of hard electronic components, we avoided using wires in our design. Instead, we used silver-plated nylon threads. In order to shield our wiring system from electromagnetic noises and make it laundering-stable, we cladded the silver threads with a three-layer fabric-based shield, comprising of polyurethane coated ripstop nylon cloth as the first layer, followed by a nanoscale (40-50 nm) coating of a hydrophobic polymer, PFDA as second, and encased within cotton piping as the last layer (shown in Figure 2c ). Our wiring system serves as lightweight and flexible interconnects between the six sewn-on electrodes and the circuit board (PCB) microcontroller (MCU).

The PhyMask system pipeline has two main components: (1) the software running on a host device to process the raw data and extract the sleep and physiological features of interest, and (2) the hardware for acquiring sensors signals and real-time Bluetooth communication. In this section, we explain these components in detail. 

We start by describing our methodology for sleep micro-events (i.e. spindle and K-complex) detection from biopotential signals, as well as sleep posture, respiratory and cardiac rhythm extraction from the pressure signals.

As discussed earlier, one of the main advantages of acquiring the EEG signal during sleep is that we can detect and analyze the sleep micro-events such as spindle and K-complex. This is important since their characteristics, e.g. frequency of their occurrence, can unveil a great amount of information regarding the cognitive state of the mind [43] . In this section, we describe the PhyMask approach for detecting spindles and K-complexes in detail. Sleep spindles are brief bursts of neural oscillations (9-16Hz, 0.5-3s) generated by the interplay of the thalamic reticular nucleus and other thalamic nuclei during NREM sleep (N2 and N3 stages) [34] (an example is shown in Figure 5c ). Like spindle, K-complex is a great hallmark of NREM sleep stage 2 and is often followed by a sleep spindle. K-complexes are generated in widespread cortical locations though they tend to predominate over the frontal parts of the brain. K-complex characteristics are very different from spindles. They consist of a brief negative sharp wave immediately followed by a positive component, creating slow-wave (0.8 Hz) and delta (1.6-4.0 Hz) oscillations of 0.5-3s in duration [30] (an example is shown in Figure 5c ). We train two different models of the same architecture for detection of spindles and K-complexes. Several classification algorithms were tested during this study, including random forests, neural networks, and support vector machines. We found the random forest classifier to provide the best performance.

In order to prepare the data for our binary random forest classifier, we first apply a 5 th order butterworth filter with a passband of 0.5-35Hz to filter out noises from the EEG streams. Then, we extract features in both time and frequency domains. Our feature extraction pipeline has two steps (summarized in Figure 6 ). First, we derive Short-time Fourier Transform (STFT), Discrete Wavelet Transform (DWT), and Empirical Mode Decomposition (EMD) features from 30-second epochs of the EEG stream and append them as additional channels to the original raw signal. In the second step, we extract statistical features from overlapping windows of the output signal of step 1. We empirically choose an input window size of 1s (125 samples) for both spindle and K-complex detection. In the following we describe the details of our feature extraction pipeline.

Short-time Fourier transform (STFT). In the literature, STFT is widely used as one of the main features for automatic spindle and K-complex detection [33, 71, 92] . STFT computation algorithm divides the input vector into overlapping shorter segments of equal length and then Fourier transform is computed separately on each segment. The output is a 3-dimensional time-series containing information regarding the frequency content of each time segment. Given the different frequency characteristic of spindle and K-complex, we extract two separate features from STFT for each event. For spindle that has a frequency content of 9-16Hz, we extract 1) sum of the powers of frequency components of the signal in 10-12 Hz, and 2) sum of the powers of frequency components of the signal within 9-16 Hz of the signal. For K-complex, which has most of its frequency content around 0.8 Hz and within 1.6-4.0 Hz, we extract 1) sum of the powers of frequency components of the signal within 0.5-1 Hz , and 2) sum of the powers of frequency components of the signal within 1.5-4.5 Hz. We then up-sample the outputs to generate a time-series of the same length as the original EEG signal and augment them with the raw EEG stream as additional channels.

Discrete Wavelet transform (DWT). To further investigate the frequency content of the EEG time-series data, we use Discrete Wavelet transform (DWT) [82] . DWT obtains low-frequency resolution and high-frequency information using long-and short-time windows, respectively. Because of this, DWT is appropriate for the analysis of the non-stationary signals such as EEG [92] . 2-level DWT and Daubechies-2 (db2) mother wavelet is used in our computation, and the second-level coefficients are taken as the output. The output is up-sampled and appended to the raw EEG and extracted STFT features.

Empirical Mode Decomposition (EMD). Empirical mode decomposition (EMD) can be used for nonlinear and non-stationary signals such as EEG [46] . In this method, a signal is divided into substatements referred to as intrinsic mode functions (IMF) and subpiece (residue). The main idea behind EMD is to locally reconstruct a signal. The reconstruction is a sum of a local trend and detail. The local trend implies the low-frequency components of the signal (residual), while the local detail (IMF) is the high-frequency parts. According to this method, a signal is recursively separated step by step. Then a certain number of IMFs and residuals are obtained depending on the content of the signal. We choose the first IMF component as the output, and then append it to the output of the previous step. Yücelbacş et al. [92] has also utilized EMD for automatic spindle detection. In their solution, they apply binary thresholding on the first IMF component derived from the EMD of a single EEG stream to detect spindle events.

Statistical feature extraction. In this step, we chunk the output of the previous step, i.e. a multi-channel signal consisting of the raw EEG, extracted STFT features, DWT features, and the EMD components, into overlapping 1-second windows and then extract statistical features including maximum, mean, median, standard deviation, sum, energy, mean-crossing, interquartile range, 10 th percentile, and 90 th percentile. Finally in order to train our random forest classifier, we use SMOTE [31] to balance the training dataset.

PhyMask can easily measure pressure changes that reflect gross motor movement of the limbs. We illustrate by looking at voltage changes of the PhyMask pressure patches when a user moves their hand. Figure 7 shows the DC-removed pressure signal for the three patches when the user is lying on their back. As the user moves their hand during time 2-4s and 9-11s, the pressure signals voltage, especially for the pressed patch, changes dramatically. Please note that the small periodic spikes on the signals correspond to the heart beats.

The head posture information during sleep can be useful in understanding which posture leads to better or worse sleep quality. For example, lying on the back is usually not recommended for sleep apnea patients [74] . Figure 8 shows the baseline signal for the P1, P2, and P3 pressure patches when a user is lying in different sleep postures for a total of 15 minutes. As it can be seen, in each sleep posture, the pressed patch has the lowest baseline. This is aligned with our expectation as the voltage being measured via the voltage divider circuit (shown in Figure 11a ) is inversely proportional to the pressure, so lower voltage means higher pressure.

Rate Estimation. In order to accurately estimate respiratory rate from the pressure sensors baseline signals (an example is shown in Figure 5b ), we first apply FFT to find the frequency bin with the highest power resulted from the respiration signal. Then, we perform band-pass filtering based around the FFT peak to avoid counting fluctuations of the second harmonic. We then count the number of peaks in one-minute-long windows of the filtered time series to obtain the respiration rate. The estimated respiration rate is updated once every half-minute (the step size is 30s). The strongest respiration signal is obtained from the patch on which the head is resting, i.e. the most pressure is sensed on.

The estimation of the heart rate is more challenging as the cardiac signal captured by the pressure patches has very a low SNR due to the mixed-in involuntary head movements, and the noise in all frequency bins. Furthermore, the amplitude of the cardiac signal can also be affected by many factors including sleep posture and user's physiological characteristics.

Generally, the signal captured by the patch under pressure in each sleep posture has better resolution in capturing the small head movements related to cardiac activity. An example is shown in Figure 5a , where the user is lying on their back and the back patch (P1) can clearly capture the heartbeats, better than the other two sensors. However, this assumption does not always hold true. Figure 9 shows another example of when the user is lying on their back and this time, the back pressure signal (P1) is quite corrupted while the right (P3) and left (P2) signals clearly detect the heartbeats. This can happen due to several reasons. Having wrinkles in the pressed patch decreases the sensor sensitivity and results in poor and noisy signal. Also, for the users who naturally have strong superficial temporal artery pulse on the temple, the heartbeat signals captured from side patches are much stronger than the back patch that only detects the subtle head motion caused by the Newtonian reaction to the influx of blood at each beat. Therefore, we have two main challenges that we need to encounter when designing our algorithm: (1) the heartbeat signal is weak and usually interfered by many sources of noises, and (2) the signal quality of patches differs across different users and sleep postures. In the following, we describe our proposed heart rate detection algorithm summarized in Figure 10 .

In order to block the DC baseline, respiration related frequency components, and higher frequency noises, we apply a 5 th order butterworth filter with a passband of 0.75-3Hz on three cardiac signals. Please note that a normal adult's resting pulse rate falls within [0.75, 2] Hz, or [45, 120] beats/min. After eliminating some of the noise, we need to isolate the heartbeat component of the three signals. In order to do this, we use Principal Component Analysis (PCA) to isolate the dominant component that contains the pulse signal. To do this, we consider each pressure patch signal stream as a separate data point and use PCA to find a set of basis vectors along which the signal has the most variation. We then select a dimension on which to project the time-series to obtain the pulse signal.

The 3 filtered BCG signals at time are represented by = [ 1( ), 2( ), 3( )]. PCA finds the principal axes of variation of the signal as the eigenvectors of the covariance matrix of the data Σ , i.e.

where Λ denotes a diagonal matrix of the eigenvalues 1, 2, and 3 corresponding to the eigenvectors in the columns of Φ , 1, 2, and 3. We finally obtain three 1-D signals ( ) by projecting onto .

We then select the most periodic component ( ) = arg max ℎ( ( )) for heart rate estimation, where the function ℎ(.) calculates the periodicity of the input signal as the percentage of the spectral power accounted for by the frequency with maximal power to the total spectral power. The heart rate is then estimated as 60 beats/min, where is the maximal frequency of ( ). Please note that we perform PCA on a 30-second window and slide it with a hop size of 10s.

We now turn to describing our hardware platform. PhyMask uses a single low-power and compact circuit board for filtering, amplification, digitization, and transmission of the raw EEG, EOG, cardiac, and respiratory signals collected from the biopotential electrodes and the pressure patches. There are three main challenges in designing such a board: first, each of these signals has different characteristics that require specific electronics circuitry design; second, all of the signal channels need to be collected at high-enough sampling rate and transmitted at the same time to ensure real-time and accurate tracking; third, the board should have very low power consumption enabling long-term usability of the system with a single battery charge. In the following, we explain the steps taken to tackle these challenges in the design of the PhyMask electronics.

As mentioned before, the board is connected to 3 biopotential electrodes (one reference and two sensors) and 3 pressure sensors. The biopotential electrodes provide the EOG and EEG signals while pressure sensors provide cardiorespiratory signals. Our board is composed of three main modules: 1) EEG and EOG signals acquisition, 2) cardiac and respiration signals acquisition, and 3) low-power wireless signal transmission. The board schematic is illustrated in Figure 11a .

Stage 1: EEG and EOG signals collection. EEG and EOG signals have very different characteristics. EOG signals, representing eye movements, have magnitudes in the orders of 100s of with a frequency range of about DC-100Hz. EEG signals however, are roughly three orders of magnitude weaker than the EOG signals and their frequency content during sleep can vary from 0.5Hz to about 30Hz.

To deal with this, we use two two-stage differential amplifiers each amplifying one biopotential signal with respect to the reference electrode. The gain of the first stage is about 30 and it outputs the EOG signal stream. We selected this level of amplification gain empirically such that the obtained EOG signal is clearly distinguishable, yet it does not saturate the next stage amplifier. This diminishes the need for a DC reject filter which consequently would have eliminated critical low-frequency contents of the EOG signal. In the second stage of the amplifier, we further amplify this signal by 2500 times and filter it to only pass through frequency content in the range of 1-50 Hz to acquire the EEG signal.

Stage 2: BCG and respirations signals collection. As discussed earlier, the fabric-based pressure patches are designed to capture the subtle ballistics and head movements caused by cardiorespiratory activities. But these two signals are very different -the pressure response of respiration is stronger than the heartbeat, and the respiration signal contains very low frequency components (typically, below 1 Hz) whereas cardiac signal is in the 1-20 Hz frequency range.

The second stage of the board receives the signal from three pressure sensors and after using a voltage divider to translate resistance changes into changes in voltage, we use a voltage follower to create a copy of the signal with low output impedance for each channel, suitable for digitization. The resultant three signals represent the posture/respiration signals of the user. To find the heartbeat instances in these signals, we then DC reject and amplify the signals up to 400 times to find the ballistics of the veins covered by the sensor. Stage 3: low-power wireless signal transmission. The last module of the board uses a low-power MCU with integrated BLE capability (nRF52811 [16]) to wirelessly transmit the acquired data to the host device. The MCU is capable of converting a maximum of 8 simultaneous analog channels into digital signals. This is less than the 10 analog channels captured by PhyMask, i.e., 2 EEG, 2 EOG, 3 cardiac, and 3 respiration. A naive solution would be to sequentially sample each channel and effectively reduce the sampling rate of all channels symmetrically. However, given the fact that the dominant frequency content of respiration is placed lower in frequency spectrum compared to other desired signals, we placed a 3:1 analog MUX to output one of the three respiration signals in each sampling round. As a result, the effective sampling rate of the respiration signal is one-third of the other physiological signals. The decisions presented above result in sampling rate of 125 samples per second for EEG, EOG, and cardiac signals, while providing 42 samples per second for the respiration signal. These sampling rates are sufficient for reconstruction of the above signals given the fact that EEG, which contains the highest frequency content among all, requires a minimum sampling rate of 60 Hz to reach the Nyquist rate and the respiration signal requires a minimum sampling rate of 2 Hz.

Our choice of sampling rate is also limited by power consumption and hardware compatibility considerations. Higher sampling rate leads to higher transmission rate for the BLE module which greatly increases the power consumption; on the other hand, some host devices have lower bound restrictions for transmission interval in their BLE connection. In our setup, the transmission interval is set to 100 ms which makes the system capable of connecting with devices with more strict restrictions and leads to average power consumption of 3.3 mw. The power consumption of the analog processing components and the MUX is negligible in comparison with the MCU. This means that with a small battery of 250 mAh, 250/1.1 capacity, PhyMask can run up to 5 days. An image of the board, which has dimensions as small as 4 cm by 5 cm, is presented in Figure 11b .

Once signals are captured and transmitted to the host device, we apply notch, median, and outlier filters in order to remove the powerline noise, and smooth out the signals. Then we filter each signal with a tailored band-pass butterworth filter (we choose 5 th order for its maximally flat passband). Examples of the PhyMask collected EEG, EOG, BCG, and respiration signals are shown in Figure 5 . 

In this section we explain the details of our user study and the ground truth labeling approach. All of these datasets were collected under Institutional Review Board approval.

Our first data collection aims to evaluate PhyMask's ability in detecting heart rate, respiration, sleep posture, gross body movement, and eye movement. For this, we asked 10 participants (average age of 27) to wear PhyMask and we recorded the output voltage in various stationary conditions. Participants varied in weight, 110-220 lb, and height, 5'1" to 6'. 4 out of 10 participants were females.

We asked our participants to lie down on their back, left, and right side, posing a sleep position that feels natural to them. We collected data for five minutes in each sleep posture. Then we demonstrated the participants a set of both smooth and jerky limb movements that are common during sleep (particularly for sleep disorders like periodic limb movement). We then repeated the first experiment and asked our participants to mimic these movements while in the sleep positions. The participants were also asked to perform random eye movements in each position for two minutes. The whole experiment led to a total of ∼30 minutes of recording from each user. Each recording consists of ten channels, six of which correspond to pressure sensing patches and four corresponding to the hydrogel fabric electrodes.

To assess the PhyMask's performance, we collect ground truth measures of the target physiological signals. For heart rate, we use a three-channel ECG measurement (2 wrists and an ankle) using the AD8232 evaluation board [8] (sampling rate of 200 Hz), and for respiration, we used a Go Direct respiration belt [4] (sampling rate of 10 Hz).

Our second data collection aims to validate PhyMask's ability in measuring the brain waves (EEG) in a more uncontrolled and naturalistic setting and to evaluate biopotential signal quality over a long time of wear. For this, we asked a participant (Male, weight = 160lbs) to wear PhyMask for five nights during sleep and we collected the signals for total of 2118 minutes (about 7 hours per night). Due to the COVID-19 precaution, the experiment took place in participant's home, providing him a safe environment to perform the study. In addition, it allowed us to collect more naturalistic sleep behavior. To evaluate how PhyMask compares to commercially-available sleep tracking wearables, we asked our participant to wear a Fitbit Charge 2 and an Oura Ring (fit size 11). The Fitbit and Oura Ring are the dominant sleep tracking wearables in the market, and this comparison allows us to gauge the benefits that PhyMask can provide due to measuring the EEG signal in addition to the physiological signals collected by these wearables.

Ground truth annotation. For ground truth, the PSG data was collected simultaneously with data from PhyMask. PSG included electroencephalographic (EEG: Fp1, Fp2, Cz, O1, O2 sites), chin electromyographic (EMG), and electrooculographic (EOG) recordings performed according to American Academy of Sleep Medicine (AASM) rules [1] (the electrodes placement is shown in Figure 13 ). The electroconductive paste (Ten20 [9]) were used to improve contact between the participant's scalp and the gold cup electrodes (with 10 mm diameter [5] ). The eight channels of data were collected via an OpenBCI Cyton amplifier [2] . In order to synchronize PhyMask's EEG signals with the PSG setup, the two PhyMask's EEG channels were also collected via the OpenBCI Cyton amplifier through the expansion Daisy Module [2]. The OpenBCI board was wirelessly connected to a laptop by the USB Dongle. The raw EEG signals were sampled at a rate of 125Hz, passed through a 60Hz notch filer, a 0.5-50Hz bandpass filter, and finally processed through the use of a mean smoothing filter to mitigate movement artifacts. Figure 12 shows a user sleeping while wearing PhyMask, sleep tracking wearables, and all the ground truth devices.

Sleep stages (Wake, N1, N2, N3, REM sleep) were scored in 30-sec epochs by certified sleep experts according to sleep scoring guideline of AASM [29] , separately for both PSG and PhyMask. In addition, the sleep micro-events i.e. spindles and K-complexes, were annotated (2094 number of spindles and 1397 number of K-complexes). For this, a sleep specialist first mark the events with Embla RemLogic PSG software [3], and then visually validate all the events.

In order to evaluate PhyMask, we break down the results into two subsections. In §6.1, we analyze the performance of our hydrogel electrodes in picking up the brain activity and eye movement signals. We present PhyMask results in detecting sleep stages as well as sleep micro-events (spindle and K-complex), and compare it with commercial wearable trackers. In §6.2, we evaluate our pressure patches performance in estimating the head posture, gross body activity, and respiration and heart rate in various sleep postures.

In this section, we explore our novel fabric hydrogel sensor's ability in measuring brain waves and eye movement patterns during the longitudinal sleep study. To the best of our knowledge, this is the first time to measure electroencephalogram (EEG) with all-fabric sensing elements during sleep. The validation of this sensor can potentially impact the next generation of EEG monitoring wearable devices.

6.1.1 Evaluating PhyMask Signal Integrity. We start by evaluating the quality of the biopotential signal obtained from the textile electrodes. We use all data for this evaluation i.e. no data is discarded due to motion artifacts or other noise issues.

We evaluate EOG signal quality using Zero-Normalized Cross-Correlation (ZNCC) metric. We use the benchmarking dataset and report the ZNCC score between the signal captured by our hydrogel electrodes and the gold cup electrodes placed at the nearest standard location, i.e. HG1 with Fp1 and HG2 with Fp2 (The sensor placement is shown in Figure 13 ). ZNCC is defined as

where is the length of ( ) and ( ), is the average of , and is the standard deviation of . Figure 14 illustrates the ZNCC score of 10 participants for lying back, right, and left scenarios. As can be seen the results are consistently good under different postures. This shows that the hydrogel electrodes precisely capture the low-frequency content of the EOG signal even when there is inconsistent pressure applied on the hydrogel electrodes caused by various sleep postures. We observe that the inconsistent pressure only affect the baseline which is removed by our band-pass filter (described in §4.1.1).

We evaluate the EEG signal quality using the coherence measure. This metric is known to identify the level of coupling in cortical pathways given its sensitivity to signal phase difference. Maximum coherence occurs when the phase difference is fixed between two signals and a near zero coherence value indicates a random phase difference between signals over time [40, 87] . Coherence is calculated as:

where ( ) is the Cross-spectral density between and , and ( ) is the autospectral density of . The goal is to evaluate how precise our electrodes can pick up the brain signals with respect to the gold standard EEG recording approach used in PSG, in an uncontrolled and naturalistic setting over a long time of wear. To understand this, we use the sleep dataset §5.2 which includes over 2118 minutes (more than 35 hours) of night sleep EEG recordings and calculate the coherence for these two groups of signals, HG1/Fp1 and HG2/Fp2, Fig. 15 . The averaged coherence measures over each frequency domain is illustrated for two groups of signals. Figure 15 illustrates the averaged coherence measures over each frequency domain. As suggested by previous studies in EEG signal analysis [40] , we label the coherence scores as following: low (0.2 to 0.3), medium (0.3 to 0.4), high (0.4 to 0.5), and very high (0.5 to 1). As it can be seen, the majority of frequency domains have very high coherence scores, confirming the maximum coupling between our fabric-based hydrogel electrodes and the standard gold-cup electrodes. Please note that for analysis of EEG signal during sleep, we are more interested in the Delta (0.5 to 4 Hz), Theta (4 to 7 Hz), and Alpha (8 to 12 Hz) sub-bands, as the Beta (12 to 25 Hz) is present when the person is awake with open eyes and is actively thinking [78] .

While Figure 15 illustrates very high coherence for the two groups of signals, it is also important to understand whether the coupling changes as time progresses. In order to investigate this factor, we have plotted the coherence score of the HG2/Fp2 pair for two different full-night sleep sessions in Figure 16 . For night #1, the consistently high coherence score (above 0.4) in lower frequency band (0.5 to 12 Hz), which includes Delta, Theta, and Alpha waves, indicates good performance and high coupling between HG2 electrode and standard Fp2 electrode. However, for night #2, the coherence degrades after time 5:00 (marked with a red arrow). The reason for this observation is that during that time period, the user had sudden body movements and roll-overs that resulted in PhyMask's displacement (about 3cm shift to the right). We see that even with this level of displacement, the average coherence measure is around 0.4 which is well within the acceptable range [40] .

6.1.2 Sleep Spindle and K-complex Detection. In this section, we evaluate 1) how well PhyMask is able to detect spindles and K-complexes, and 2) how PhyMask compares to PSG-based spindle and K-complex detection; PSG has higher-dimension EEG signals captured by standard electrodes from various points on the scalp, so it provides a measure of the best-case performance. Fig. 16 . Time series of the coherence score of the right hydrogel electrode/Fp2 pair for two different nights. While the coherence is consistently high for night #1, for night #2, it degrades after 5:00 due to sudden body motions. In order to compare the performance of PhyMask with PSG in sleep micro-event detection, we train our classifiers explained in §4.1.1 separately on PhyMask EEG data and the gold standard PSG. The results of running the two sets of classifiers using K-fold (k = 20) cross validation is summarized in Table 2 and 3. Due to the highly imbalanced nature of the data, we use sensitivity and specificity measures for evaluating the performance of the micro-event detectors as also common in the literature [73, 79] .

Our results are very promising and show that: (1) PhyMask accurately detects spindle and K-complex events with higher than 0.8 sensitivity, specificity, and accuracy scores, and (2) PhyMask performs as precise as PSG in spindle and K-complex detection. This means that PhyMask is capable of recording high fidelity EEG signals that contain the micro-sleep events such as spindles and K-complexes.

6.1.3 Validation of PhyMask against Polysomnography for Sleep Stage Tracking. In this section, we evaluate the quality of the PhyMask signal against gold-standard PSG for the purpose of sleep stage tracking. In order to do this, both PhyMask and PSG sleep data (more than 35 hours of sleep data) have been annotated by sleep experts into five sleep stages, i.e., N1, N2, N3 (Deep sleep), REM, and Awake. Due to the low number of N1 stage instances in our dataset, we grouped both N1 and N2 stages as Light sleep. In order to quantify the level of agreement for annotated sleep stages between PhyMask and PSG, we use Cohen's kappa [32] defined as follows:

where is the relative observed agreement among raters, and is the hypothetical probability of chance agreement, using the observed data to calculate the probabilities of each observer randomly seeing each category. If the raters are in complete agreement then = 1. If there is no agreement among the raters, = 0. It is possible for the statistic to be negative, which implies that there is no effective agreement between the two raters or the agreement is worse than random. It is well-known that there exists inevitable variability and disagreements between human experts on epoch ratings -Wang et al. [89] found kappas of 0.72-0.85 between two human raters and kappas of 0.82-0.85 between well-performing raters. Therefore, a Cohen's kappa of 0.8 is considered to show strong agreement once we take into account such variabilities. Figure 17 summarizes, for each sleep stage, the Cohen's kappa coefficients over all five sleep sessions (total number of epochs = 4236). The overall performance of PhyMask reaches the state of the art with a median Cohen's kappa of 0.77. For most of the sleep stages, PhyMask agrees very well with the PSG. This high level of agreement between PhyMask and PSG further validates the quality of the measured EEG signal and shows that PhyMask successfully captures information that is critical for sleep stage tracking.

The common misclassified stage in PhyMask data is Awake. In order to better understand when sleep stage misclassifications happen, we calculate the confusion matrix (Figure 18) , where each data point in the confusion matrix represents a 30-second epoch. As can be see, the awake stage is mostly confused with the REM stage. Since, these stages have overlapping frequency bands (13Hz-30Hz), the main distinguishing signal between the two is the EOG. During the REM stage, as its name suggests, fast rapid eye movements occur as opposed to slow rolling eye patterns that happen during the Awake/drowsiness stage [75] . PhyMask can only capture the horizontal eye movements; however, human raters are trained on conventional data that includes both horizontal and vertical eye movement patterns. We believe that this might have led to lower classification accuracy in this case, and we hypothesize that in the future, training a classifier on the PhyMask data should significantly help with this.

In this section we compare PhyMask with two other commercially-available sleep stage tracking devices i.e. Fitbit Charge 2 and Oura Ring. Figure 19 illustrates the extracted sleep stages of one night from the three devices with respect to the PSG ground-truth system.

We compare the accuracy, precision, recall, and f1 score measures of PhyMask, Oura, and Fitbit, calculated as Figure 20 summarizes the results for all of the sleep stages over all nights data. PhyMask outperforms the other two devices in terms of all four metrics, which demonstrates the advantage that PhyMask offers in accurate sleep stage tracking by leveraging its capability in recording the EEG signal.

We also calculate the cohen's kappa measures and confusion matrices for Oura and Fitbit ( Figure 21 ). Oura (median kappa = 0.54) and Fitbit (median kappa = 0.51) have much lower kappas than PhyMask (median kappa = 0.77). Low accuracy of these sleep trackers in detecting sleep stages has also been observed by previous studies [35, 36] . Since we do not have access to the raw data captured by Oura and Fitbit sleep tracking algorithms, we cannot fully analyze their behaviour. However, if we look at the confusion matrices, we find that Oura has a very low recall for the Awake stage. This is because Oura does not provide the micro-awake events after sleep onset in their provided sleep hypnogram on the dashboard. They instead seem to be reported as the closest sleep stage and that is why the predicted labels for Awake events by Oura are distributed almost evenly among all the stages. Fitbit, on the other hand, has a noticeable bias towards classifying sleep sessions as the Light sleep stage. 

In this section, we evaluate the performance of our novel pressure-based method for detecting the physiological variables of interest. For this purpose we use our benchmarking dataset 5.1.

Gross body movement. As discussed earlier, the PhyMask pressure patches are designed to be sensitive to capture the subtle head movements caused by heart pulses and respiration. Therefore, the gross body movement, which is an important marker for sleep tracking and sleep disorder diagnosis, can easily be detected by the embedded pressure sensors.

In order to evaluate this, we asked our participants to mimic both smooth and jerky limb movements that are common during sleep in each sleep posture ( §5.1). The median variance of baseline signal voltage of the three patches for stationary, hand movement, and leg movement scenarios across all sleep postures and over 10 participants are illustrated in Figure 22a . As can be seen, the variance difference between the limb movement scenarios and the stationary scenario is very distinct. We can also see that the signal corresponding to the hand movement scenario has lower variance compared to the leg movement scenario since people in general use more force to move their legs as opposed to their hands due to the legs physical characteristics such as weight and length which results in applying more pressure on the patches. As a result, even naive thresholding on the variance of the pressure patch signals can identify different types of gross body movement events.

In order to understand how gross motor activity affect each pressure patch in different sleep postures, we calculate the variance of each patch baseline voltage during limb movement periods over each sleep posture. The median voltage across all participants is shown in Figure 22b . As can be seen, the under-pressure patch has the highest variance meaning the higher sensitivity to detect the body movements. An interesting point is that in both lying on right and left scenarios, the variance of the back patch is as high as the pressed patch. This is because, based on our observation, the participants tend to swing their whole body a bit back and forth while We can see that pressure patches easily capture information on gross body movement.

doing the activities, specifically during the leg movements. These motions greatly affect the back patch which results in high variance.

Head posture. In addition to gross motor activity, PhyMask can also provide information about head posture. This information can be useful in understanding which posture leads to better or worse sleep. Head posture can be unambiguously estimated using PhyMask by using the signal baseline across pressure patches (shown in Figure 8 ). Figure 23 summarizes the voltage changes averaged across all participants for each sleep posture. In each posture, the voltage change is calculated for all the patches (back, right, and left) with the following formula,

where 0 is the mean baseline voltage of patch when the participant is in a seated position and there is no pressure applied to the pressure patch, and represents the mean baseline voltage of patch in the specified sleep posture.

As expected, the pressed pressure patch in each sleep posture has the highest voltage difference comparing with other pressure patches. We find that a simple decision tree can easily identify posture with 100% accuracy across all subjects.

Note that the negative voltage occurs when the pressure applied to the corresponding patch in a specific sleep posture is lower than what it is in the seated position. One example is the right and left patches in the lying on back sleep posture ( Figure 23 ). This is quite intuitive, since in the seated position the right and left pressure patches are resting on the ears and more pressure is applied to them comparing to when the person is lying on back and the side patches are sort-of loose. Figure 24 shows the performance of PhyMask in estimating the breathing rate for all of the participants across three sleep postures. We see that respiration can be accurately captured by the PhyMask pressure sensor simply by looking at periodic pressure changes on the head. The respiration metrics are very good and the median error is generally about 1 resp/minute. 6.2.3 Heart Rate Estimation. The heart rate estimation error is also very good. As can be seen in Figure 25 , the median heart rate error over all participants across all the sleep postures is about 1.7 beats/min, that is within the acceptable error margin (= 5 bpm) for heart rate measurement [42] . The only participant with high error is #7. Based on the ground truth values, we find that this participant has generally weaker heart beats that result in more subtle head oscillations. We believe that in the future, we can further increase the sensitivity of our pressure patches in order to better detect these small movements.

In Figure 26 , we breakdown the PhyMask performance in estimating heart rate for different sleep postures across all the participants. The results are very promising and shows that PhyMask is robust in all the scenarios.

Breaking down the contributions of the heart rate estimation model components. As described in §4.1.3, in order to measure heart rate, we first apply PCA on three pressure patches and then detect the best component that holds the majority of the frequency content corresponding to the heart rate. But one question is how much each of these steps contribute to the overall heart rate estimation performance. To understand this, we compare three variants of our model. First we have the full model, PhyMask. For the second model, "no-PCA + best patch", we remove the PCA step that is responsible for decomposing the patch signals into sub-signals in order to isolate pulse. Therefore, the best component detection algorithm is directly applied to the pressure patches, and hence the heart rate is derived from the best selected patch. Lastly, we look at the performance, when we directly choose the highest peak in the pressed pressure patch derived fft as the estimated heart rate. This model can help us better understand whether the pressed patch is the best patch for accurate heart rate estimation and how effective our best component detection algorithm is. We call this model "no-PCA + pressed patch". Figure 27 summarizes the results. We see that removing PCA ("no-PCA + best patch") increases the mean heart rate error about 1.3 beats/minute. This demonstrates that PCA is successful in isolating heart beats across the three patches. In the case of "no-PCA + pressed patch", where we also remove the best component detection step and directly measure the heart rate from the pressed pressure patch, the mean heart rate error increases about 1 beat/minute. This validates that the under-pressure patch in each sleep posture is not necessarily the most sensitive one. This is also consistent with our observation as described in §4.1.3.

At the end of our user study, we asked our participants to fill out a questionnaire regarding the PhyMask comfort and their preferences. The questions and the users' answers are presented in Figure 28 . The results show that the majority found PhyMask quite comfortable. 9/10 users were interested in tracking their physiological signals during the sleep. None of the users had used sleep masks before, while 6/10 users had been using wearable tracking devices on a daily basis. While all the participants preferred PhyMask over PSG, only 3 participants preferred PhyMask over Fitbit/Oura, none of the whom owned any wearable tracking devices. An explanation for this can be that participants who already are used to wearing smart wearables, are also less willing to change their habit.

In conclusion, we introduce a new all-textile system, the PhyMask, for monitoring a number of important sleep signals including EEG, EOG, respiration, heart rate, and gross body movement. Despite a plethora of commercial and research prototype sleep tracking solutions, we lack a reliable and comfortable solution that can continually monitor the whole range of sleep markers that are useful for clinical-grade sleep monitoring without impacting sleep. PhyMask bridges this gap and shows that all of these signals can be monitored by solely using soft textile-based sensors by leveraging a combination of soft hydrogel electrodes and sensitive textile-based pressure sensors. Such a system can be useful to enable high-quality clinical-grade sleep monitoring at home. Our results are very promising and demonstrate that we can measure a number of advanced sleep markers such as spindles and K-complexes in addition to sleep stages. We can also provide robust measures of physiological features from an unconventional place, i.e., head, with high accuracy across different sleep postures.

Cyton + Daisy Biosensing Boards

Embla RemLogic PSG Software for sleep and micro-events annotations

Go Direct Respiration Belt

Cup Electrodes

Murata bcg mems sensor

Neuroon Open -Sleep tracking solution

Single Lead Heart Rate Monitor AD8232

Actiwatch Spectrum Activity monitor | Philips Healthcare

Fitbit official site for activity trackers

Muse™ -Meditation Made Easy with the Muse Headband

Oura Ring: Accurate Health Information Accessible to Everyone

American Clinical Neurophysiology Society guideline 2: guidelines for standard electrode position nomenclature

Smart Homes That Monitor Breathing and Heart Rate

Novel hydrogel-based preparation-free EEG electrode

Sleep disorders and sleep deprivation: an unmet public health problem

Defining requirements and related methods for designing sensorized garments

Detecting Pulse from Head Motions in Video

Camera-based system for contactless monitoring of respiration

The AASM manual for the scoring of sleep and associated events. Rules, Terminology and Technical Specifications

The human K-complex represents an isolated cortical down-state

SMOTE: synthetic minority over-sampling technique

A coefficient of agreement for nominal scales

An Automatic Sleep Spindle detector based on WT, STFT and WMSD

Sleep spindles: an overview

A validation study of Fitbit Charge 2™ compared with polysomnography in adults

The Sleep of the Ring: Comparison of theŌURA Sleep Tracker Against Polysomnography

Design and implementation of a noncontact sleep monitoring system using infrared cameras and motion sensor

K-complex-induced seizures in autosomal dominant nocturnal frontal lobe epilepsy

REM sleep behavior disorder and dementia: cognitive differences when compared with AD

Time estimation and beta segregation: An EEG study and graph theoretical approach

Electrooculography by wearable graphene textiles

Heart rate estimation using facial video: A review

Sleep spindle density predicts the effect of prior knowledge on memory consolidation

PressION: An All-Fabric Piezoionic Pressure Sensor for Extracting Physiological Metrics in Both Static and Dynamic Contexts

The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis

Sleep quality and elevated blood pressure in adolescents

Cortical reactivations during sleep spindles following declarative learning

Monitoring a Person's Heart Rate and Respiratory Rate on a Shared Bed Using Geophones

Monitoring a Person's Heart Rate and Respiratory Rate on a Shared Bed Using Geophones

Sleep oscillation-specific associations with Alzheimer's disease CSF biomarkers: novel roles for sleep spindles and tau

Validity of a commercial wearable sleep tracker in adult insomnia disorder patients and good sleepers

Inflammatory aspects of sleep apnea and their cardiovascular consequences

Phyjama: Physiological Sensing via Fiber-Enhanced Pyjamas

Wearable Multi-Biosignal Analysis Integrated Interface With Direct Sleep-Stage Classification

Role of sleep duration and quality in the risk and severity of type 2 diabetes mellitus

Noncontact vision-based cardiopulmonary monitoring in different sleeping positions

Development of an EOG-Based Automatic Sleep-Monitoring Eye Mask

Development of an EOG-based automatic sleep-monitoring eye mask

Novel dry polymer foam electrodes for long-term EEG measurement

Tracking Vital Signs During Sleep Leveraging Off-the-Shelf WiFi

Tracking vital signs during sleep leveraging off-the-shelf wifi

Micro-and macrostructure of sleep EEG

Breath rate monitoring during sleep using near-IR imagery and PCA

A validation study of the WHOOP strap against polysomnography to assess sleep

Validation of Fitbit charge 2 and Fitbit Alta HR against polysomnography for assessing sleep in adults with obstructive sleep apnea

Boosting Fine-Grained Activity Sensing by Embracing Wireless Multipath Effects

Sleep stage estimation method using a camera for home use

Fully textile, PEDOT: PSS based electrodes for wearable ECG monitoring systems

A wearable health care system based on knitted integrated sensors

Automated sleep spindle detection using novel EEG features and mixture models

DoppleSleep: A Contactless Unobtrusive Sleep Sensing System Using Short-Range Doppler Radar

A fuzzy neural network approach for automatic K-complex detection in sleep EEG signal

The undervalued potential of positional therapy in position-dependent snoring and obstructive sleep apnea-a review of the literature

A review of signals used in sleep analysis

W!NCE: Unobtrusive Sensing of Upper Facial Action Units with EOG-Based Eyewear

Graphene coated cotton fabric as textile structured counter electrode for DSSC

Chapter 1 -Introduction. In EEG Brain Signal Classification for Epileptic Seizure Disorder Detection

Benchmarking matching pursuit to find sleep spindles

Symptoms of depression in individuals with obstructive sleep apnea may be amenable to treatment with continuous positive airway pressure

Age affects sleep microstructure more than sleep macrostructure

The discrete wavelet transform: wedding the a trous and Mallat algorithms

Multilayer sweat-absorbable textile electrode for EEG measurement in forehead site

Sleep microstructure around sleep onset differentiates major depressive insomnia from primary insomnia

The link between short sleep duration and obesity: we should recommend more sleep to prevent obesity

Origin and significance of the cyclic alternating pattern (CAP)

Coherence, Phase Differences, Phase Shift, and Phase Lock in EEG/ERP Analyses

Fully-automatic camera-based pulse-oximetry during sleep

Evaluation of an automated single-channel sleep staging algorithm

Sleep disturbance, cognitive decline, and dementia: a review

Multiparameter sleep monitoring using a depth camera

Automatic detection of sleep spindles with the use of STFT, EMD and DWT methods