An automatic apnea screening algorithm for children Expert Systems With Applications 48 (2016) 42–54 Contents lists available at ScienceDirect Expert Systems With Applications journal homepage: www.elsevier.com/locate/eswa An automatic apnea screening algorithm for children Sebastián A. Ríos∗, Lili Erazo Business Intelligence Research Center, Department of Industrial Engineering, University of Chile, Av. Republica 701, Santiago, Chile a r t i c l e i n f o MSC: 00-01 99-00 Keywords: Screening algorithms Apnea classification Apnea screening Ubiquitous health system a b s t r a c t Sleep Disordered Breathing (SDB) is a group of diseases that affect the normal respiratory function during sleep, from primary snoring to obstructive sleep apnea (OSA) being the most severe. SDB can be detected using a complex and expensive exam called polysomnography. This exam monitors the sleep of a person during the night by measuring 21 different signals from an Electrocardiogram to Nasal Air Flow. Several au- tomatic methods have been developed to detect this disorder in adults, with a very high performance and using only one signal. However, we have not found similar algorithms especially developed for Children. We benchmarked 6 different methods developed for adults. We showed empirically that those models’ per- formance is drastically reduced when used on children (under 15 years old). Afterwards, we present a new approach for screening children with risk of having SDB. Moreover, our algorithm uses less information than a polysomnography and out performs state-of-the-art techniques when used on children. We also showed em- pirically that no signal alone is a good SDB screening in children. Moreover, we discover that combinations of three signals which are not used in any other previous work are the best for this task in children. © 2015 Elsevier Ltd. All rights reserved. c i m ( R p n e r ( i n A a 2 c b t a a 1. Introduction Several health systems from different countries have agreed on the need of tackling public health by the means of a more preven- tive, personalized and anticipatory health service. This requires to en- couraging patients and caregivers to make them protagonists of their healthcare, assuming responsibility to keep themselves on controlled states without collapsing healthcare centers. In Chile, we have a severe case of air pollution, which produce episodes where hospitals collapse during autumn and winter. This is why; Health Ministry is developing several programs that aim to treat patients at home. But to do so, we need to know exactly a patient, for example, what is their normal biomedical signals values (mean and standard dev.) or what happens with humidity during winter inside their home or other environmental conditions. Based on specific data and other information from the patients and knowledge extracted from experts we can develop an expert system similar to Echeverría, Jimenez-Molina, and Ríos (2015) that aid patients and caregivers to take better decisions (even from their homes). This article focuses in the creation of an automatic screening method for Sleep Disordered Breathing (SDB) which are a group of chronic diseases that affect the normal respiration function during the night. These can affect people at any age and are due to different ∗ Corresponding author. Tel.: +56 968353577. E-mail addresses: srios@dii.uchile.cl, sebastian@rios.tv (S.A. Ríos), lerazo@ing.uchile.cl (L. Erazo). o h e P http://dx.doi.org/10.1016/j.eswa.2015.11.013 0957-4174/© 2015 Elsevier Ltd. All rights reserved. auses: in newborns and young children they are related to congen- tal defects or premature birth; in older children and adults they ay be related to obesity, morphological causes, and hypertension Goodwin et al., 2003a; 2003b; Levy, Bonsignore, & Eckel, 2009; osen, Palermo, Larkin, & Redline, 2002). The diseases -from least to most severe- considered as SDB are: rimary snoring, upper airway resistance, and obstructive sleep ap- ea (OSA). For example, a newborn with OSA can experience short pisodes of apnea, total absence of airflow during night and live a elatively normal life, while a severe OSA may lead to sudden death Goldstein et al., 2004). Symptoms of SDB are abnormal day sleepiness, sudden naps dur- ng the day (for example, at a red light while driving), general tired- ess, fatigue, trouble sleeping, and other related diseases (de la Luz lonso Álvarez et al., 2008). It has been shown that in children SDBs re closely related to obesity and learning problems (Goodwin et al., 003a). The diagnosis of SDB is accomplished through a clinical study alled polysomnography (PSG) which collects over 20 different iomedical signals during sleep, including: electrocardiography, elec- roencephalography, electromyography, plethysmography, oronasal irflow, chest movement, abdominal movement and leg movement mong others. Several automatic methods have been developed in the form f expert systems to detect these disorders in adults, with a very igh performance (Álvarez-Estévez & Moret-Bonillo, 2009; De Chazal t al., 2003; Driver et al., 2011; Jarvis & Mitra, 2000; Khandoker, alaniswami, & Karmakar, 2009). However, we showed empirically http://dx.doi.org/10.1016/j.eswa.2015.11.013 http://www.ScienceDirect.com http://www.elsevier.com/locate/eswa http://crossmark.crossref.org/dialog/?doi=10.1016/j.eswa.2015.11.013&domain=pdf mailto:srios@dii.uchile.cl mailto:sebastian@rios.tv mailto:lerazo@ing.uchile.cl http://dx.doi.org/10.1016/j.eswa.2015.11.013 S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 43 b c t n u o f h s p E n h s a t n t 2 b n M s p s e o e r b l b d J S d n c d o b r c b i b w s t E c w w c a t s A w 7 a e a 3 c H t e m g m m y Erazo and Ríos (2014) that those models’ performance is drasti- ally reduced when used in children (under 15 years old). This time we present a new algorithm able to classify children into wo groups: the group at risk of having an SDB, and the group with o risk (or very little risk) of having an SDB. Moreover, our algorithm ses less information than a polysomnography and surpasses state- f-the-art techniques when used on children. This is very important actor since in order to develop a non-invasive solution (to be used at ome, for example) we need to reduce the amount of signals used by creening methods. We showed empirically that no signal alone can be a good OSA redictor in children and also we showed that only three signals: MG-Chin, EOG and Leg Movement are very good predictors. We eed to remark this result, since until today, none of these signals ave been used on their own or in combination with other signals for creening in any documented study reviewed in this research. This work is based on data collected by Pablo E. Brockmann M.D. nd his research team in the Sleep Study Center of Clinical Hospi- al of Catholic University of Chile. This dataset consists in 78 whole ight polysomnographies from patients under 15 years old. This is he largest dataset of this characteristics used for OSA screening. . Related work Our main interest is to develop an algorithm that can distinguish etween OSA and non-OSA individuals with the least amount of sig- als from the PSG. In adults, several researchers (Álvarez-Estévez & oret-Bonillo, 2009; Driver et al., 2011; Flemons et al., 2003) have hown that some signals from PSG have enough predictive power to erform this task. In fact, one-signal screening methods have been uccessfully tested in adults and children (Roche et al., 2003; Tsai t al., 2013), but those methods still require attending personnel and vernight dedicated systems. Most of them also required a medical valuation afterwards. Automated classification methods aim to avoid unnecessary esource consuming screening methods. The most important contri- ution to this respect was made by the Computers in Cardiology Chal- enge of 2000. In that work the task was to automatically tag, minute y minute, a single ECG signal as OSA or non-OSA and get to a final iagnosis: OSA or non-OSA for every record (De Chazal et al., 2003; arvis & Mitra, 2000; Khandoker et al., 2009; Mendez et al., 2007). ome of these methods reached a precision of 100% in the binary iagnosis and over 85% in minute-by-minute tagging. Unfortunately, Table 1 Automated OSA screening approaches tested in our previous work (Erazo & Ríos, 2014). Author, Year Brief description R Jarvis and Mitra (2000) Apnea diagnosis based on ECG signal. Features used derived from spectral analysis. Classification based on threshold criteria. A De Chazal et al. (2003) Apnea diagnosis based on ECG signal. Features extracted using black-box methods. Classification methods: linear discriminant, quadratic discriminant. A Mendez et al. (2007) Apnea binary diagnosis based on ECG signal. Features extracted: derived signals (EDR, HRV) coefficients. Classification method: K-Nearest Neighbour. A Álvarez-Estévez and Moret-Bonillo (2009) A fuzzy reasoning is used to detect apneic events; Air flow signal and Oxygen Saturation signals are used Se Khandoker et al. (2009) Apnea binary diagnosis (OSA+, OSA-) based on ECG signal. Features extracted: Wavelet Transform coefficients. Classification method: Support Vector Machine. A Driver et al. (2011) Validation of MediByte (portable monitor) as screening method. Se one of these studies on automatic classification was performed on hildren. Besides, most of the algorithms were trained and tested on atabases specially designed for this task. In particular, models tested n the Computers in Cardiology 2000 database can not be compared, ecause this database has been preprocessed to obtain clean, but not ealistic data. This is why we performed a benchmark with real data from 78 hildren and compared the results among them. This was published y Erazo and Ríos (2014) and a summary of methods tested is shown n Table 1. The best classifiers from all approaches tested were those ased on: a Support Vector Machine (SVM), an Artificial Neural Net- ork (ANN). Besides we implemented a Logit classifier to evaluate a imple model, though it is not in the literature. We also selected the hree signals that reported best results in the literature, which are: lectrocardiography (ECG or ECGI channel), Air Flow (Patient Airflow hannel) and Oxygen Saturation (SpO2 channel). Afterwards, we pre-processed ECG, Air Flow and SpO2 with avelet transform to extract features. All algorithms were trained ith all 14 features generated. Then we trained the models with a ross-validation approach with 70% of the dataset to train the models nd 30% to test; and finally, experiments were performed 30 times o generate the final benchmark results computing several measures hown in Section 3.3. The best performing models were the ANN applied over oronasal ir Flow signals, with Sensitivity = 84.36%. The second best model as the SVM with Air Flow signal that resulted in a Sensitivity = 5.64%. Both classifiers were far from the 90% sensitivity considered s the clinical minimum for a successful classifier. We demonstrated xperimentally that state-of-the-art models for OSA screening in dults are not good enough to be used in children. . Model construction methodology After performing our benchmark by Erazo and Ríos (2014), we ould not find a good predictor for OSA+ and OSA- in children’s data. owever, models using Air Flow signal had an outstanding sensi- ivity, over 80% and regular accuracy (in the Neural Network mod- ls). ECG based models, on the other hand showed high specificity, eaning that they have the ability to detect healthy people. This sug- ests that a combination of these signals may lead to a successful odel. This section describes a novel approach to this task. From a purely athematical point of view, signal selection is performed in order esults Comments cc=100% Tested on Computers in Cardiology 2000 database. 75 records tagged minute-by-minute. All registers corresponding to adults. cc=100% Tested on Computers in Cardiology 2000 database. 75 records tagged minute-by-minute. All registers corresponding to adults. cc=85,5%, Sens=83,9%, Spec=88,5% Tested on Physionet Apnea ECG Database (50 registers) tagged as OSA+ or OSA-. ns=87%, Spec=89% Tested on a 12 patients database (Sleep Heart Health Study), it does not diagnose OSA, it only detects apneic events. All registers corresponding to adults. cc=100% Tested on three combined databases: Sleep Research Unit Database, Physionet Apnea ECG Database and Saint Vincent’s University Hospital/University College Dublin Sleep Apnea Database. All 125 registers corresponding to adults. ns=80%, Spec=87% This monitor requires a clinician to perform the diagnosis. 44 S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 Table 2 State-of-the-art benchmark results’ summary (Erazo & Ríos, 2014). Method WT func. ECG Air flow SpO2 Sens Spec Acc Sens Spec Acc Sens Spec Acc [%] [%] [%] [%] [%] [%] [%] [%] [%] SVM Db1 8,3 82,1 50,0 75,6 26,3 54,2 9,7 82,6 50,9 Db9 2,7 88,2 51,0 75,4 28,7 55,1 15,0 79,7 51,6 Haar 10,0 79,7 49,4 70,8 32,7 54,2 13,3 77,2 49,4 Sym8 5,3 85,1 50,4 75,1 27,0 54,2 12,3 82,1 51,7 NN Db3 34,3 60,3 49,0 84,4 19,0 55,9 30,0 75,1 55,5 Db4 36,3 63,9 51,9 80,5 21,7 54,9 19,7 77,7 52,5 Haar 25,7 65,9 48,4 80,3 26,3 56,8 38,7 60,0 50,7 Sym1 28,7 67,7 50,7 80,3 24,3 55,9 41,0 63,3 53,6 Logit Db3 44,6 54,5 50,2 18,9 74,6 43,7 44,5 51,4 48,4 Db4 41,9 53,0 48,2 19,3 78,9 44,9 55,1 61,9 59,0 Haar 43,6 58,4 51,8 39,3 58,1 46,3 47,0 55,8 52,1 Sym7 43,9 51,0 47,9 19,1 75,6 43,4 52,2 57,0 55,0 A A a t f r E D t 2 ( l p t s f f c e M V E C D fi a f f m 3 t to construct a minimal signal model able to satisfactorily screen the pediatric population. 3.1. Data understanding All children included in this study came to the Sleep Center be- cause their parents or doctors had concerns about the quality of their breathing during sleep. Patients’ parents had to answer a stan- dard questionnaire in order to evaluate the severity of the potential disorders. Only children considered as a risk population underwent polysomnographic test. Each patient spent a whole night in the hospital where 20 or 21 different biomedical signals were recorded (approximately 8 h of records),1 including Electroencephalography (11 channels: EEG F3- 1,EEG C3-A1, EEG P3-A1, EEG O1-A1, EEG F4-A2, EEG C4-A2, EEG P4-A2, EEG O2-A2, EEG A1-A2, EEG Fp1-A1, EEG Fp2-A2), Electro- cardiography (ECG or ECGI channel), Electroculography (EOG Right and EOG Left), Electromiography (EMG Chin), Air Flow (Patient Air- flow channel) and Oxygen Saturation (SPO2 channel) among others. PSG was collected in Sleep Study Center with an Alice® 5 Diagnostic Sleep System from Respironics (Philips) and transformed using the corresponding software (Alice® Sleepware) to the generally accepted format for biomedical time series: European Data Format (EDF). Afterwards each test was manually scored by Dr. Pablo E. Brock- mann or one of his fellows according to AASM (American Academy of Sleep Medicine) criteria (General, 2005). Apnea, hypopnea, arousals, leg movement, and any other event of interest were reported. Data corresponds to 78 complete night PSG from 78 different pa- tients of ages from 2 to 16 years old (mean ± SD: 9.7 ± 3.6 years). Sixty-four percent of them are girls. According to AASM criteria, children are diagnosed with OSA with an Apnea Hypopnea index equal or greater than 1 (General, 2005). Using this specification, 43% of the PSG correspond to children with OSA. No PSG was excluded because of other diseases presented in patients. It is important to mention that most studies of OSA screening have small databases, no more than 20 registers. Even though some bigger databases are available, in child data, this is the biggest database doc- umented (as far as this research went). 3.2. Feature extraction The feature extraction method selection was based on the most referenced one in literature, as explained by Erazo and Ríos (2014). 1 originally records were almost 9 hours long, but after data cleaning; approximately 8 h remained. n A c ccording to this criteria, the Wavelet Transform method was dopted. Wavelet Transform is used in 1D non–stationary signals to process ime series as the ones collected by PSG. It allows for the collecting of requency information contained in the time series. Also, there is no equirement of the frequency stability, thus it is frequently used for CG feature extraction. Several wavelets were tested in order to extract features. aubechies and Symlet of levels from 1 to 10 were tested, in addi- ion to the Haar function. This means that every signal was processed 1 times under each of these 21 functions of a Wavelet Transform Daubechies1, Daubechies2, Daubechies3, and so on). To process the dataset, the Wavelet Toolbox for Matlab was se- ected. With this objective, EDF’s had to be transformed into a com- atible Matlab format. Each signal processed with Wavelet Transform was decomposed o its 14th detail level, obtaining 14 time series for each decompo- ition function, i.e. 14 time series for ECG processed with the Haar unction, 14 time series for ECG processed with Daubechies of level 1 unction, and so on. In order to reduce data and extract features, three features were alculated for each time series, these were: mean, variance and en- rgy; calculated as follows: ean: x̄ = 1 N N∑ i xi (1) ariance: σ 2 = N∑ i (x̄ − xi )2 (2) nergy: E = N∑ i x2i (3) oefficients obtained from the preprocessing stage were named b1mean1, Db1var1, Db1ene1 for the mean, variance and energy of the rst detail level for Daubechies1 WT function. The same process was pplied for every WT function and to every signal. This way, 21 dif- erent databases were generated for every signal –one for each WT unction–, each of them containing 42 features: mean1, var1, ene1, ean2, var2, ene2 up to mean14, var14, ene14. .3. Evaluation measures In order to compare models we will compute well known evalua- ion measures. The main point is to describe the strengths and weak- esses of the models (Fig. 1). Indicators selected for these tasks are Sensitivity, Specificity and ccuracy (commonly used to assess clinical tests performance); Pre- ision, Recall and F-measure (to assess the classification performance S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 45 Feature Extraction Feature Selection Minimum Feature Selection PSG Data Multi-signal Models Training SVM ANN Logit Sub set of features (70% data) Multi-signal Models Test SVM ANN Logit Sub set of features (30% data) Cross-Validation Evaluation Sensitivity Specificity Precision Recall FAccuracy Fig. 1. Summary of Experimental Methodology. o R S A P F I h c fi i c g a p 3 u A p p a Q s a v t d n i c s M f algorithms). They will be calculated as follows: ecall = Sensitivity = T P T P + F N (4) pecificity = T N T N + F P (5) ccuracy = T P + T N T P + T N + F P + F N (6) recision = T P T P + F P (7) = 2 ∗ T P 2 ∗ T P + F N + F P (8) nterpretation of these indicators is based on the ability to detect un- ealthy individuals. This is because it is more important to correctly lassify sick children as sick than healthy children as healthy. Sensitivity is the ability to detect unhealthy individuals, Speci- city is the ability to detect healthy people, Accuracy is the abil- ty to correctly classify, that is, how many children were correctly lassified. On the other hand, Precision is the proportion of the ill individuals roup that are correctly classified. Finally, the F-measure shows how ccurate is the model, is in a range of 0 to 1, where 1 is the best model ossible. .4. ECG feature extraction To extract relevant features from an ECG signal, it is important to nderstand how it is analyzed by clinicians. Fig. 2 shows a sample of an ECG signal corresponding to the 0011823 patient. A regular ECG wave can be characterized according to some oints. As seen in Fig. 3, points P, Q, R, S and T describe a complete ulse of an ECG. The relevant characteristics for respiratory functions are associ- ted with the QRS complex. This is the name of the area formed by , R and S points. Commonly the features are extracted from derived ignals from the ECG recording. These signals are: Heart Rate Vari- bility (HRV) and ECG Derived Respiratory Signal (EDR). HRV corresponds to the time series constructed from R–R inter- als; this is the time between successive R points. EDR corresponds o the time series constructed from QRS amplitude. This is the vertical istance between Q and R points. Before extracting features, all signals had to be normalized to ig- ore the variance associated with natural variations among different ndividuals. While a certain heart rate might be normal to a person, it ould be considered accelerated for another, so normalization is the tandard proceeding when working with biological signals. Common features extracted from HRV and EDR signal are: ean: x̄ = 1 N N∑ i xi (9) 46 S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 Fig. 2. ECG sample. Fig. 3. ECG wave characteristics. Fig. 4. Electrode positions for an EEG. 3 E a n Variance: σ 2 = 1 N N∑ i (x̄ − xi )2 (10) Also, some time domain features were extracted from the HRV signal. In order to do this a new time series had to be constructed, corresponding to the successive differences of the HRV signal. These features were: Root Mean Square of Successive Differences: = √ N∑ i (xi − xi−1 )2 (11) Standard Deviation of Successive Differences: = √ 1 N N∑ i (x̄ − xi )2 (12) With all these transformations the resulting feature set to character- ize the ECG signal is: meanHRV, varHRV, meanEDR, varEDR, RMSSD and SDSD. .5. EEG feature extraction The EEG performed during a PSG has 11 signals as output: EGF p1 − A1, EEGF p2 − A2, EEGA1 − A2, EEGF 3 − A1, EEGF 4 − A2, EEGC3 − A1, EEGC4 − A2, EEGP3 − A1, EEGP4 − A2, EEGO1 − A1 and EEGO2 − A2. Each of them characterized by the position of the elec- trode in the head, where A1 corresponds to the left side, and A2 to the right side, as seen in Fig. 4. Not all patients present every signal. In small children (under 2 years old) not all the electrodes are placed on the head, mainly be- cause they have smaller heads, so 31 of the registers do not contain the signals EEGA1 − A2, EEGF p1 − A1, EEGF p2 − A2, EEGO1 − A1 y EEGO2 − A2. Although, some techniques propose to process them ll together (by adding the signals or applying Principal Compo- ents Analysis for example), the selected approach was to process S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 47 Fig. 5. EEG sample corresponding to patient A0001945. Fig. 6. Spectrogram sample corresponding to patient A0001945. t s p s m r t a w o i u t a s t f c c t c u f f f W t F a t n e 3 g m p p e d m s f hem separately for two reasons: first, clinicians analyze them eparately, and second, if there is a combination that allows for rocessing them all together it should be reflected in the feature election stage. This is, if the signals are giving redundant infor- ation, the feature selection will detect this effect and eliminate edundancy. When a PSG is processed, clinicians look into EEG signals to tag ime windows according to the sleep stage the patient is in. For ex- mple in Fig. 5 is easy to distinguish two stages. So, the main task ith these signals is to detect variations in the wave frequencies in rder to characterize different stages. What matters most is to determine how much a wave varies dur- ng night, for which a methodology was developed. First, a spectrogram was constructed. Spectrograms are frequently sed to analyze sound waves or pixels. They are a graphic represen- ation of the variety of frequencies in a wave as time varies. On the X-axis a spectrum of frequencies is represented. On the Y- xis the time is divided in small windows for analysis, and the color cale shows how strong is the presence of a frequency in a specific ime window. If columns are analyzed independently, the variation of a specific requency can be observed. Based on this, the features to be extracted haracterize the number of times a big variation occurs, that is, a hange in the sleep stage is suspected. If smaller time windows are used, smaller changes can be de- ected, but this effect is not desired, since changes in sleep stages are haracterized by big variations, so only eight time windows will be sed for the analysis. In order to do this, an auxiliary time series is defined as the dif- erence of a frequency from one time window to the next. This is per- ormed for every frequency considered in the spectrogram (Fig. 6). The main idea is to detect significant variations, so the relative dif- erences were calculated as the difference divided by the initial value. ith this, a new time series was constructed for each signal. Later, his signal was filtered to make it easier to detect big variations. Fig. 7 shows the processed signal corresponding to the sample in ig. 5, every frequency analyzed is shown in a different color, the vari- tion in time is represented graphically. This means, peaks represent he big changes indicating a change in the sleep stage is suspected. Finally, features extracted from each EEG signal correspond to the umber of local minimum and the number of local maximum from very frequency filtered signal. .6. Abdominal and thoracic effort feature extraction Abdominal and Thoracic Effort Signals correspond to a band that oes around the abdomen and chest to register a patient’s move- ents while sleeping. The main idea is to detect peaks of movement, that is, when a atient has a regular respiratory function, no abnormality should ap- ear in the signal. In Fig. 8 for example, an abnormality appears at the nd of both signals (time ∼ 8500). In order to detect peaks wavelet ecomposition was used for its ability to show them in the signal. The ethodology implemented to extract features from the signal con- iders the mean, standard deviation and energy of the signals derived rom the decomposition. 48 S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 Fig. 7. Processed EEG signal corresponding to patient A0001945. Fig. 8. Abdominal Thoracic Signal (blue) and Abdominal Effort Signal (red) corresponding to patient A0000678. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.) Fig. 9. Different Wavelet Decompositions (blue) for the Thoracic Effort Signal (red) corresponding to patient A0000678. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.) t l 3 a As said in the previous section, many families of functions can be used with this decomposition. A visual inspection was done to select the signal that best fits the objectives of this task. Fig. 9 shows three of the wavelet functions tested. Although all functions seem to detect variations equally, Daubechies 3 had a better performance amplifying small variations, as the one seen at the end of the time window shown in the figure. This is why he function selected to process both signals was Daubechies 3 of evel 4. .7. Air flow feature extraction The Air Flow Signal registers the air movement through the nose nd the mouth. So, to detect the presence of OSA, one must detect S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 49 Fig. 10. Air Flow Signal corresponding to patient A0000739. Fig. 11. (a) Leg1 Signal corresponding to patient A0001484. (b) Leg2 Signal corresponding to patient A0001484. (c) Legs Signal corresponding to patient A0011947. w o a W f t 3 m e p m w d o e f s d 3 f m m b p t e a t t 3 t t d hen an abnormal flow occurs, this is, the complete absence of flow r a big decrease in it. Fig. 10 shows a complete night record. The task is to detect the flat reas in the signal. The same task was approached by Erazo and Ríos (2014) using a avelet Transform where the features extracted were used as input or one-signal models. This resulted in high performance models. Therefore the same strategy was followed and the selected func- ion was Daubechies 3 of level 14. .8. Leg movement feature extraction Of the 78 patients, 36 had only one signal that recorded their leg ovement during sleep. Forty-one of them had only one record for ach leg, this is two signals per patient to record leg movement. One atient had no record. As seen on Fig. 11 both legs are highly correlated, so no lost infor- ation is suspected in the case of only one record. Yet, both signals ere included in the analysis. If the information results are redun- ant, the Feature Selection Stage should detect it. The characterization of the signal had to represent the variation f the leg movement to detect abnormal movement or big movement pisodes. As before, the Wavelet Transform has the ability to extract these eatures from the signal. After testing a variety of functions, Daubechies 8, of level 6 was elected. As in previous cases, mean, variance and energy of every etail level were calculated, obtaining a set of 18 features. .9. Body position feature extraction Body Position signal detects the changes of position of the body, rom supine (back facing down) to prone (chest facing down). The ain idea is to detect change in positions, as a measure of the move- ent during the night. The sensor used for this task assigns a num- er to each position. It is a nominal variable that detects changes in osition. In order to reflect the number of times a patient moves during he night an auxiliary signal was constructed as the successive differ- nces of the signal. Therefore only changes in position would appear s different from zero as seen in Fig. 12. With this, the feature ex- racted was an index of the number of changes divided by the sleep ime. .10. Electromyogram feature extraction The Electromyogram signal (EMG) registers the electrical poten- ial generated by muscle cells, in a PSG in particular is used to regis- er chin movement during sleep. Chin movement is important in the iagnosis of SDB, and OSA, because respiration may occur through 50 S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 Fig. 12. (a) Body Position Signal corresponding to patient A0001460. (b) Auxiliary signal derived from Body Position Signal for patient A0001460. Fig. 13. EMG Chin Signal corresponding to patient A0002197. 3 o i a f t t l e t p i i f m nose or mouth, thus chin movement is used as an indicator of mouth movement. On the other hand, tension in the chin might indicate snoring or another abnormalities during sleep, like bruxism for example. As other signals that register movement, signal processing was made by a Wavelet Transform. The same selection process was per- formed, the chosen function was Daubechies 7 of level 8, resulting in 24 features (Fig. 13). 3.11. Electro-oculogram feature extraction The Electro-oculogram (EOG) registers the movement of the eyes by placing electrodes around them. The result of this exam is one sig- nal for each eye. So, in a complete PSG, two signals correspond to EOG, and they are tagged as EOG Right, and EOG Left for the right eye, and the left eye respectively. Even though these are separated signals, a high correlation is ex- pected, since eye movements are not independent from each other in most cases. In Fig. 14 we see the correlation between both left and right eye of patient A0006225. The same correlation is observed in every record from this signal, yet every signal was processed because the feature extraction process should be able to detect these highly correlated features. In order to be consistent in the treatment of signals, all movement signals were processed with Wavelet Transform. The function used for the EOG feature extraction was Daubechies 7 of level 8. .12. Pulse feature extraction The pulse signal as predictor for OSA prevalence has been used in ther studies. As expected, an abnormal pulse rate might be a good ndicator of an apnea episode due to the lack of oxygen generated as consequence. In particular, Noehren et al. (2010) use four time series derived rom the pulse signal: • APRD: Absolute Pulse Rate Decrease • APRI: Absolute Pulse Rate Increase • RPRD: Relative Pulse Rate Decrease • RPRI: Relative Pulse Rate Increase For the purpose of this study, only the first two were used. The pulse signal was normalized to avoid variation among pa- ients (see Fig. 15(a)). Then a Butterworth filter was used to eliminate he noise from the signal and detect easily peaks of local minima and ocal maxima (see Fig. 15(b)). The APRD length corresponded to the number of peaks found, and ach value was calculated as the difference of a local minimum and he local maximum immediately before (see Fig. 15(d)). The same rocedure was applied to construct the APRI (see Fig. 15(c)), calculat- ng the difference between a local maximum and the local minimum mmediately before. After this, the results are two time series. The decision is to extract, rom both of them: mean, standard deviation, median, maximum and inimum. S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 51 Fig. 14. EOG Sinals of Left (blue) and Right (Red) ayes corresponding to patient A0006225. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.) Fig. 15. (a) Pulse signal corresponding to patient A0000294. (b) Filtered pulse signal. (c) Absolute Pulse Rate Increase (APRI) derived from pulse signal. (d) Absolute Pulse Rate Decrease (APRD) derived from pulse signal. 3 l c s v b v s a i t s e w 3 r p a a t I F l l d ( d a w a .13. Snore feature extraction The snore signal is used to diagnose not only OSA, but other SDB’s, ike primary snoring. Although very useful for SDB diagnosis, the final lassification into OSA+ and OSA- groups can not rely only on this ignal. Snoring is measured by a sensor located on the throat that detects ibration. As snoring is produced by an obstruction of the airway, vi- ration is produced, and measured this way. The resulting signal is normalized since the relevant aspects are ariations in the signal. As output for the PSG, a signal from 0 to 1 is hown. The goal when processing this signal was to determine when an bnormal vibration episode occurred. The signal provided by the PSG s already normalized, so an abnormal episode was considered when he signal crosses over 0.7 or under 0.3 (blue lines on Fig. 16). This election was based on preliminary analysis of the signal behavior. The features extracted correspond to the number of abnormal pisodes up and abnormal episodes down. The resulting feature set as: Snoreup and Snoredown. .14. Oxygen saturation feature extraction PSG also include a sensor that measures the blood’s oxygen satu- ation during the whole night. Usually saturation measures are done eriodically. Therefore a PSG is one of the few clinical tests that allows Saturation Signal. Evidently this is one of the most important signals to determine diagnosis, since it is directly related to apnea episodes. A desatura- ion episode is a clear symptom of an abnormal respiratory function. n particular, it might be the perfect indicator of an apnea episode. ig. 17 shows a time window of the Oxygen Saturation Signal col- ected by a PSG from patient A0001183. Many indicators are used for processing this signal. The most se- ected and relevant for this study are: DEI (Desaturation Event In- ex), defined as a saturation signal drop in four or more points on a scale from 1 to 100); and ODI (Oxygen Desaturation In- ex), also known as CT90 (Cumulative Time 90). Corresponds to the mount of time the patient has a saturation under 90%. DEI and ODI ere selected because of their previous use in continuous positive irway pressure devices (CPAP) tests and previous experiences in 52 S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 Fig. 16. Snore Signal corresponding to patient A0000884. Fig. 17. Oxygen Saturation Signal corresponding to patient A0001183. w b f c t P s diagnosing apnea episodes (Chang, Wu, & Cao, 2012; Chung et al., 2012). So, as result, the feature set was: DEI and ODI for each signal. 4. Feature selection Resulting from the Feature Extraction process, 355 features are used to resume the information collected in a PSG. Table 3 summa- rizes the features resulting from the previous section. The hypothesis is that not all the information is needed to perform a high quality screening. If this is true, a less resource-consuming clinical test may be developed to pre-diagnose children with sus- pected OSA. The first step to accomplish this is to select features. The selec- tion should imply a reduction in the signals needed to perform a screening. A filter approach was selected to perform the Feature Selection. This allows the use of the same subset with all three models and al- lows comparison under the same conditions as shown by Erazo and Ríos (2014). Embedded methods were discarded due to their dependency on the selected model. Therefore, different models would have ended up ith different feature subsets. Wrapper methods were discarded also, ecause they did not allow for the control of the number of resulting eatures in the subset. Before starting the selection, data quality was checked. Features orresponding to the Pulse signal were discarded since only 6 pa- ients had complete Pulse records. The selection procedure was based on correlation analysis and rincipal Component Analysis. The step by step methodology is de- cribed as follows: 1. First. Principal Component Analysis (PCA) was computed over all the Features included, using Varimax rotation criterion. The main objective was to extract variables that explain most of the vari- ance in order to incorporate as much information as possible in the modeling section. 2. Second. The first five components resulting from Factorial Anal- ysis were analyzed, as they explained 89.9% of the variance, and the sixth component added only 2% to the variance explanation. 3. Third. Variables eigenvalues were extracted, and only variables with an eigenvalue of 0.05 (5%) or more for the first five principal components were kept. 4. Fourth. As mentioned, all features corresponding to selected sig- nals were kept. Resulting from these steps, features corresponding S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 53 Table 3 Summary of resulting features from a complete PSG. Signal Features extracted Total ECG meanHRV , varHRV , meanEDR , varEDR , RMSSD and SDSD. 6 EEG (11) max1 , max2 , max3 , max4 , max5 , max6 , max7 , min1 , min2 , min3 , min4 , min5 , min6 and min7 . 154 Abdominal effort Db3mean1 , Db3var1 , Db3ene1 , Db3mean2 , Db3var2 , Db3ene2 , Db3mean3 , Db3var3 , Db3ene3 , Db3mean4 , Db3var4 y Db3ene4 . 12 Thoracic effort Db3mean1 , Db3var1 , Db3ene1 , Db3mean2 , Db3var2 , Db3ene2 , Db3mean3 , Db3var3 , Db3ene3 , Db3mean4 , Db3var4 y Db3ene4 . 12 Air flow Db3mean1 , Db3var1 , Db3ene1 , Db3mean2 , Db3var2 , Db3ene2 , Db3mean3 , Db3var3 , Db3ene3 , Db3mean4 , Db3var4 , Db3ene4 , Db3mean5 , Db3var5 , Db3ene5 , Db3mean6 , Db3var6 , Db3ene6 , Db3mean7 , Db3var7 , Db3ene7 , Db3mean8 , Db3var8 , Db3ene8 , Db3mean9 , Db3var9 , Db3ene9 , Db3mean10 , Db3var10 , Db3ene10 , Db3mean11 , Db3var11 , Db3ene11 , Db3mean12 , Db3var12 , Db3ene12 , Db3mean13 , Db3var13 , Db3ene13 , Db3mean14 , Db3var14 and Db3ene14 . 48 Leg movement (2) Db8mean1 , Db8var1 , Db8ene1 , Db8mean2 , Db8var2 , Db8ene2 , Db8mean3 , Db8var3 , Db8ene3 , Db8mean4 , Db8var4 , Db8ene4 , Db8mean5 , Db8var5 , Db8ene5 , Db8mean6 , Db8var6 and Db8ene6 . 36 Body position CI 1 EMG Db7mean1 , Db7var1 , Db7ene1 , Db7mean2 , Db7var2 , Db7ene2 , Db7mean3 , Db7var3 , Db7ene3 , Db7mean4 , Db7var4 , Db7ene4 , Db7mean5 , Db7var5 , Db7ene5 , Db7mean6 , Db7var6 , Db7ene6 , Db7mean7 , Db7var7 , Db7ene7 , Db7mean8 , Db7var8 and Db7ene8 . 24 EOG (2) Db7mean1 , Db7var1 , Db7ene1 , Db7mean2 , Db7var2 , Db7ene2 , Db7mean3 , Db7var3 , Db7ene3 , Db7mean4 , Db7var4 , Db7ene4 , Db7mean5 , Db7var5 , Db7ene5 , Db7mean6 , Db7var6 , Db7ene6 , Db7mean7 , Db7var7 , Db7ene7 , Db7mean8 , Db7var8 and Db7ene8 . 48 Pulse APRDmean , APRDstd , APRDmedian , APRDmax , APRDmin , APRImean , APRIstd , APRImedian , APRImax andAPRImin . 10 Snore Snoreup and Snoredown 2 Oxygen saturation DEI and ODI 2 TOTAL 355 Table 4 Summary of selected features. Signal Features selected Total Leg 1 Db8mean1 , Db8mean2 , Db8var2 , Db8mean3 , Db8mean4 , Db8mean5 , Db8var5 and Db8mean6 . 8 Leg 2 Db8mean3 , Db8mean4 , Db8mean5 and Db8var5 . 4 Legs Db8mean1 , Db8var1 , Db8mean2 , Db8mean3 , Db8mean4 and Db8mean5 . 6 EOG right Db7mean1 , Db7mean2 , Db7mean3 , Db7mean4 , Db7mean5 , Db7mean6 , Db7mean7 and Db7var7 . 8 EOG left Db7mean1 , Db7var1 , Db7mean2 , Db7mean3 , Db7var3 , Db7mean4 , Db7mean5 , Db7mean6 , Db7mean7 , Db7var7 and Db7mean8 . 11 EMG chin Db7mean1 , Db7var1 , Db7mean2 , Db7mean3 , Db7mean4 , Db7mean5 , Db7mean6 , Db7mean7 , Db7var7 , Db7mean8 and Db7var8 . 11 TOTAL 51 S 5 t p s d a c r s Table 5 SVM performance using the feature subset from PSG. Model Sensitivity Specificity Accuracy Precision Recall F-measure SVM 100,00% 100,00% 100,00% 100,00% 100,00% 100,00% Table 6 NN performance using the feature subset from PSG. Model Sensitivity Specificity Accuracy Precision Recall F-measure NN 28,67% 100,00% 28,67% 100,00% 28,67% 39,91% Table 7 Logit Regression performance using the feature subset from PSG. Model Sensitivity Specificity Accuracy Precision Recall F-measure LOGIT 47,30% 100,00% 47,30% 100,00% 47,30% 62,69% 6 m w p s c m b 7 a g f p c s i b to the following signals formed the preliminary subset: EMG Chin, EOG (Left and Right), Snore and Legs. Preliminary subset had 129 features. 5. Fifth. Correlation Matrix was computed over the preliminary sub- set. 6. Sixth. Every pair of signals was analyzed. If a correlation of over 0.6 was found, only one of the pair of signals was kept. 7. Seventh. Resulting from these steps, 51 features remain in the subset. This is the definitive Features Subset. Table 4 shows the resulting subset of features from the Feature election process. . Classification models Models used to classify patients into the two groups defined are he same as those used by Erazo and Ríos (2014). These are: Sup- ort Vector Machine, Artificial Neural Networks and Logit Regres- ion. Also, the same validation methodology was implemented, as escribed in Section 3.3. Thirty different training and testing balanced sets were created nd used to train and test every model. Performance metrics were alculated each time. The resulting performance corresponds to the average of the met- ics resulting from every iteration (this is, the result from one training et and one testing set). . Results Tables 5–7 describe the results obtained from each classification odel separately. Support Vector Machines showed an outstanding performance hen the minimal signal set are used as input. This result is unex- ected because, as far as this research went, no other published re- earch used EMG, EOG and leg movement as input signals. In fact, linicians use these signals for auxiliary information, and never as the ain source of information for diagnosis. The other models showed poorer performance compared to SVM, ut they also showed unexpected Precision results. . Discussion Having as a starting point the previous results presented by Erazo nd Ríos (2014), where it is shown that no signal alone can be a ood predictor for OSA in children when using black-box methods or feature extraction. As a consequence, it can be inferred that a more hysiological approach to feature extraction may lead to good quality lassification methods. Evidence in adults and some studies conducted in children howed good-quality screening methods were mainly based on phys- ological feature extraction methods. Most of these studies also, were ased on one-signal only. The few of them that were based on more 54 S.A. Ríos, L. Erazo / Expert Systems With Applications 48 (2016) 42–54 A p t P a o U R Á C C D D E E F G G G G J K L d M N R R T than one signal did not show any criterion for the signal selection beyond clinicians criteria. Always having present the goal of finding the minimum feature set that can effectively screen for OSA. This is why the approach pre- sented here describes a novel method for OSA screening: Signal se- lection was based only on statistical and optimization criteria. We discover that three signals: EMG-Chin, EOG and Leg Move- ment combined are the best predictor for OSA in children. Sur- prisingly none of these signals have been used on their own or in combination with other signals for screening in any documented study reviewed in this research. Also, all of this signals are collected with ordinary electrodes, widely available on the market. This fact allows us to think of this algorithm as an actual screening method for future research in devel- oping countries where resources are scarce, and OSA consequences in infant populations are largely overlooked. Our approach is able to screen OSA with 100% precision was devel- oped using signals collected by polysomnography and the pertinent information needed was reduced using data mining techniques, so only three signals are needed to perform a high quality screening. Another result of this work, a high quality repository of polysomnographic children data tagged by qualified clinicians is available for further testing and model construction. It is very important to consider that today is a world wide ten- dency the development of ubiquitous health (u-health) expert sys- tems that allow the monitoring of patients even at home (Echeverría et al., 2015). Most methods for screening OSA use oxygen saturation and ECG signals (see Table 1); which need very good quality sen- sors (expensive) in order to obtain good quality measures. This is a barrier to develop a system for use at home or in third world coun- tries. However, the sensors required to measure our approach signals (EMG-Chin, EOG and Leg Movement) are more simple in construc- tion, price and allow to have good quality data for a u-health expert system. 8. Recommendations and future work For signals like ECG or Air Flow, widely used in screening methods for adults, the results showed no relevance or correlation with the target variable in the feature selection process (see Section 4). Clini- cians base their diagnosis on these signals, although they use all the information available from a polysomnography. The intuitive reason- ing lead them to think that these signals should appear as relevant in the feature selection. This fact may be evidence that extracted fea- tures are not representative enough of the information contained in the signal. Therefore new approaches to feature extraction should be sought after with more adequate methods for each signal. Some research devote most of the work to performing feature extraction correctly, therefore a comparison of different feature extraction methods is rec- ommended in order to correctly represent signals for the feature se- lection process. If this algorithm, or any other, is intended to be implemented as a valid screening method, it must meet minimum requirements. First, is has to be precise. Second, it has to be easier to perform than the gold-standard, less resource consuming and independent of clinician supervision. The proposed method in this research actually meets some of these requirements. But there is a third requirement, not less important: clinicians must trust the screening test in order to actually prescribe it. As previously mentioned, as far as this research went, no docu- mented work uses any of the resulting signals (EMG Chin, EOG, Leg Movement) alone or combined with others for prediagnosis. This is mainly because these signals are not the main source of information in a regular diagnosis process, so is necessary to check doctor’s opin- ion and their disposition to prescribe this kind of clinical test. cknowledgment This work was financed by CONICYT - IDeA FONDEF programme, roject code: CA13i-10300. Also, authors would like to thank the con- inuous support of “Instituto Sistemas Complejos de Ingeniería” (ICM: -05-004- F, CONICYT: FBO16). Last but not least, we would like to thank Dr. Pablo Brockmann nd his team at Clinical Hospital from Pontifical Catholic University f Chile and Ivan Videla at Business Intelligence Research Center from niversity of Chile for his comments on the manuscript. eferences lvarez-Estévez, D., & Moret-Bonillo, V. (2009). Fuzzy reasoning used to detect apneic events in the sleep apnea-hypopnea syndrome. Expert Systems with Applications, 36, 7778–7785. hang, L., Wu, J., & Cao, L. (2012). Combination of symptoms and oxygen desaturation index in predicting childhood obstructive sleep apnea . International journal of pe- diatric otorhinolaryngology, 77(3), 365–371. hung, F., Liao, P., Elsaid, H., Islam, S., Shapiro, C. M., & Sun, Y. (2012). Oxygen desatu- ration index from nocturnal oximetry: a sensitive and specific tool to detect sleep- disordered breathing in surgical patients. Anesthesia & Analgesia, 114, 993–1000. e Chazal, P., Heneghan, C., Sheridan, E., Reilly, R., Nolan, P., & O’Malley, M. (2003). Automated processing of the single-lead electrocardiogram for the detection of obstructive sleep apnoea. IEEE Transactions on Biomedical Engineering, 50, 686–696. river, H. S., Pereira, E. J., Bjerring, K., Toop, F., Stewart, S. C., Munt, P. W., & Fitzpatrick, M. F. (2011). Validation of the medibyte type 3 portable monitor com- pared with polysomnography for screening of obstructive sleep apnea. Canadian Respiratory Journal: Journal of the Canadian Thoracic Society, 18, 137. cheverría, M., Jimenez-Molina, A., & Ríos, S. A. (2015). A semantic framework for con- tinuous u-health services provisioning. Procedia Computer Science, 60, 603–612. razo, L., & Ríos, S. A. (2014). A benchmark on automatic obstructive sleep apnea screening algorithms in children. Procedia Computer Science, 35, 739–746. lemons, W. W., Littner, M. R., Rowley, J. A., Gay, P., Anderson, W. M., Hudgel, D. W., et al. (2003). Home diagnosis of sleep apnea: A systematic review of the litera- turean evidence review cosponsored by the american academy of sleep medicine, the american college of chest physicians, and the american thoracic society. CHEST Journal, 124, 1543–1579. eneral, A. (2005). International classification of sleep disorders, 2nd edn.: Diagnostic and coding manual. westchester, illinois: American academy of sleep medicine, 2005. broughton r. pathological fragmentary myoclonus, intensified sleep starts and hypnagogic foot tremor: three unusual sleep related disorders. Sleep, 28, 113– 121. oldstein, N. A., Pugazhendhi, V., Rao, S. M., Weedon, J., Campbell, T. F., Goldman, A. C., et al. (2004). Clinical assessment of pediatric obstructive sleep apnea. Pediatrics, 114, 33–43. oodwin, J. L., Babar, S. I., Kaemingk, K. L., Rosen, G. M., Morgan, W. J., Sherrill, D. L., et al. (2003a). Symptoms related to sleep-disordered breathing in white and hispanic children. the tucson childrens assessment of sleep apnea study. CHEST Journal, 124, 196–203. oodwin, J. L., Kaemingk, K. L., Fregosi, R. F., Rosen, G. M., Morgan, W. J., Sherrill, D. L., et al. (2003b). Clinical outcomes associated with sleep-disordered breathing in caucasian and hispanic children. the tucson childrens assessment of sleep apnea study (tucasa). Sleep Disordered Breathing, 26, 587–591. arvis, M., & Mitra, P. (2000). Apnea patients characterized by 0.02 hz peak in the mul- titaper spectrogram of electrocardiogram signals. In Computers in cardiology 2000 (pp. 769–772).IEEE. handoker, A. H., Palaniswami, M., & Karmakar, C. K. (2009). Support vector machines for automated recognition of obstructive sleep apnea syndrome from ecg record- ings. Information Technology in Biomedicine, IEEE Transactions on, 13, 37–48. evy, P., Bonsignore, M., & Eckel, J. (2009). Sleep, sleep-disordered breathing and metabolic consequences. European Respiratory Journal, 34, 243–260. e la Luz Alonso Álvarez, M., Santos, J. T., Guevara, J. C., Martínez, M. G., Pascual, L. R., Ba nuelos, J. L. V., et al. (2008). Reliability of home respiratory polygraphy for the diagnosis of sleep apnea-hypopnea syndrome. analysis of costs. Archivos de Bron- coneumología (English Edition), 44, 22–28. endez, M. O., Ruini, D. D., Villantieri, O. P., Matteucci, M., Penzel, T., Cerutti, S., et al. (2007). Detection of sleep apnea from surface ecg based on features extracted by an autoregressive model. In Engineering in medicine and biology society, 2007. embs 2007. 29th annual international conference of the ieee (pp. 6105–6108).IEEE. oehren, A., Brockmann, P. E., Urschitz, M. S., Sokollik, C., Schlaud, M., & Poets, C. F. (2010). Detection of respiratory events using pulse rate in children with and with- out obstructive sleep apnea. Pediatric pulmonology, 45, 459–468. oche, F., Pichot, V., Sforza, E., Duverney, D., Costes, F., Garet, M., et al. (2003). Predict- ing sleep apnoea syndrome from heart period: a time-frequency wavelet analysis. European Respiratory Journal, 22, 937–942. osen, C. L., Palermo, T. M., Larkin, E. K., & Redline, S. (2002). Health-related quality of life and sleep-disordered breathing in children. Sleep, 25, 657–666. sai, C.-M., Kang, C.-H., Su, M.-C., Lin, H.-C., Huang, E.-Y., Chen, C.-C., et al. (2013). Usefulness of desaturation index for the assessment of obstructive sleep apnea syndrome in children. International journal of pediatric otorhinolaryngology, 77(8), 1286–1290. http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0001 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0001 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0001 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0001 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0002 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0002 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0002 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0002 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0002 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0003 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0003 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0003 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0003 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0003 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0003 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0003 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0003 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0004 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0004 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0004 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0004 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0004 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0004 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0004 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0004 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0005 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0005 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0005 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0005 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0005 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0005 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0005 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0005 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0005 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0006 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0006 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0006 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0006 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0006 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0007 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0007 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0007 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0007 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0008 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0008 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0008 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0008 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0008 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0008 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0008 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0008 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0009 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0009 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0010 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0010 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0010 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0010 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0010 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0010 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0010 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0010 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0011 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0011 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0011 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0011 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0011 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0011 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0011 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0011 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0012 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0012 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0012 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0012 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0012 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0012 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0012 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0012 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0013 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0013 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0013 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0013 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0014 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0014 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0014 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0014 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0014 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0015 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0015 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0015 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0015 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0015 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0016 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0016 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0016 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0016 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0016 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0016 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0016 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0016 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0017 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0017 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0017 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0017 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0017 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0017 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0017 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0017 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0018 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0018 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0018 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0018 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0018 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0018 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0018 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0018 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0019 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0019 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0019 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0019 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0019 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0019 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0019 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0019 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0020 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0020 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0020 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0020 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0020 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0020 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0021 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0021 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0021 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0021 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0021 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0021 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0021 http://refhub.elsevier.com/S0957-4174(15)00779-4/sbref0021 An automatic apnea screening algorithm for children 1 Introduction 2 Related work 3 Model construction methodology 3.1 Data understanding 3.2 Feature extraction 3.3 Evaluation measures 3.4 ECG feature extraction 3.5 EEG feature extraction 3.6 Abdominal and thoracic effort feature extraction 3.7 Air flow feature extraction 3.8 Leg movement feature extraction 3.9 Body position feature extraction 3.10 Electromyogram feature extraction 3.11 Electro-oculogram feature extraction 3.12 Pulse feature extraction 3.13 Snore feature extraction 3.14 Oxygen saturation feature extraction 4 Feature selection 5 Classification models 6 Results 7 Discussion 8 Recommendations and future work Acknowledgment References