key: cord-350793-bftztg0e
authors: Nizami, Shermeen; Green, James R.; McGregor, Carolyn
title: Implementation of Artifact Detection in Critical Care: A Methodological Review
date: 2018-04-30
journal: IEEE reviews in biomedical engineering
DOI: 10.1109/rbme.2013.2243724
sha: 
doc_id: 350793
cord_uid: bftztg0e

Artifact Detection (AD) techniques minimize the impact of artifacts on physiologic data acquired in Critical Care Units (CCU) by assessing quality of data prior to Clinical Event Detection (CED) and Parameter Derivation (PD). This methodological review introduces unique taxonomies to synthesize over 80 AD algorithms based on these six themes: (1) CCU; (2) Physiologic Data Source; (3) Harvested data; (4) Data Analysis; (5) Clinical Evaluation; and (6) Clinical Implementation. Review results show that most published algorithms: (a) are designed for one specific type of CCU; (b) are validated on data harvested only from one Original Equipment Manufacturer (OEM) monitor; (c) generate Signal Quality Indicators (SQI) that are not yet formalised for useful integration in clinical workflows; (d) operate either in standalone mode or coupled with CED or PD applications; (e) are rarely evaluated in real-time; and (f) are not implemented in clinical practice. In conclusion, it is recommended that AD algorithms conform to generic input and output interfaces with commonly defined data: (1) type; (2) frequency; (3) length; and (4) SQIs. This shall promote (a) reusability of algorithms across different CCU domains; (b) evaluation on different OEM monitor data; (c) fair comparison through formalised SQIs; (d) meaningful integration with other AD, CED and PD algorithms; and (e) real-time implementation in clinical workflows.

hysiologic signals exhibit trends, dynamics and correlations reflecting the complexity of underlying patient physiology [1, 2] . Continuous monitoring of physiologic data assists clinicians in making diagnoses and prognoses in Critical Care Units (CCU), including Intensive Care (ICU), Paediatric Intensive Care (PICU), Neonatal Intensive Care Units (NICU), and the Operating Room (OR) [3] . Clinical Event Detection (CED) techniques analyze these data to identify clinically significant events and early onset indicators of various pathophysiologies as in [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] . Parameter Derivation (PD) techniques derive clinically useful low frequency parameters from high frequency input data as in [17] [18] [19] [20] [21] [22] [23] .

Artifacts are extraneous signals with randomly varying amplitudes, frequencies and duration that interfere with physiologic signals acquired in clinical settings [24] . Longitudinal studies [25] [26] [27] [28] infer that Original Equipment Manufacturer (OEM) patient monitors have relatively simplistic built-in data preprocessing for Artifact Detection (AD). In this paper, the term AD encompasses one or more of the following mechanisms: (a) identification, detection or annotation of artifacts, (b) elimination of artifacts or artifact-laden data and (c) suppression or filtering of artifacts e.g., by using ensemble averaging or adaptive filtering [29] . The terms noise and cleaning have been used in publications as alternatives for the terms artifacts and AD respectively. OEM monitors typically come with a black box approach to preprocessing artifacts in physiologic data. Clinicians debate the reliability of OEM monitor data as they deduce that some built-in algorithms employ a reductionist approach that oversimplifies complex human physiology [30] . Artifacts can mimic physiologic data [31] [32] [33] , adding to the challenge of distinguishing between the two. As a result, data logged by OEM monitors remains impacted by artifacts [34] [35] [36] . This increases false alarm rates in monitors [36] [37] [38] , which leads to staff desensitization [30, 39, 40] . Clinicians cannot rely on analysing artifact-laden monitor data [41] , which in the past has resulted in incorrect diagnoses [33, 42] ; unnecessary therapy; surgery and iatrogenic diseases [32] . Independent research groups have developed a variety of post processing techniques to address the problem of artifacts in OEM monitor data. Fig. 1 shows the AD configurations reviewed in this paper. The purpose of AD is identical in any configuration, i.e., to assess and enhance the quality of physiologic data. Fig.  1 depicts Signal Quality Indicators (SQI) among other output variables. Latest AD research shows increasing interest in devising SQIs [43] [44] [45] [46] . This paper conducts a critical review of the development and utility of SQIs.

This methodological review introduces six thematic taxonomies for a critical analysis of the state-of-the-art in AD. The objective is to synthesize the status of clinical evaluation and implementation of AD in various CCUs. Reviewed publications are listed in Table I . Section II discusses past reviews of AD. Section III briefly describes research methodology followed by detailed development of a unique taxonomy for each of the six themes. Review results are thematically synthesized in Table II . Section IV concludes the review by highlighting open research problems. It also provides specific recommendations for new research directions in promoting implementation of AD in real-time clinical workflows.

Historically, AD has been reviewed with significantly different scope and context from this paper, which makes this review a novel contribution in this research space. Borowski et al. [47] have conducted a comprehensive review on alarm generation in medical devices. Their primary focus is

Care: A Methodological Review ergonomics and human factors engineering, although, they briefly synthesize some AD algorithms and, as a result, recommend using multivariate alarm systems. Schettlinger et al. [48] have largely reviewed their own research in developing univariate filters for outlier, trend and level shift detection in various ICU data types. They extensively describe filter development and compare advantages and disadvantages of different filter designs. Chambrin [49] infers that multivariate techniques can reduce false alarm rates in ICUs.

Numerous publications, such as [37, 43, 48, [50] [51] [52] , provide comparative numeric summaries of performance metrics of AD algorithms. This review neither reiterates algorithmic details nor performance comparison as that falls outside its scope. Takla et al. [28] note that while AD techniques developed by independent researchers may have higher specificity than built-in algorithms in OEM monitors, extensive studies are required to evaluate their accuracy prior to real-time implementation in clinical environments. Siebig et al. [38] demonstrate agreement amongst clinical staff, including intensivists, that integrative monitoring through data fusion can potentially yield better results as compared to simpler univariate threshold detection methods. Imhoff et al. [37, 52] emphasize that methodological research is needed for integrating multivariate AD algorithms in real-time clinical systems. However, they do not present a conceptual framework to conduct such research.

Papers of interest were located in Scopus, IEEE Xplore and PubMed databases using the keywords artifact, artefact, artifact detection, alarm, physiologic monitoring, intensive care, neonatal intensive care, operating room and patient monitors amongst others. No constraints were applied on the publication year. Google Scholar was also used. Literature was then methodologically reviewed using six thematic taxonomies developed in this section: (1) Critical Care Unit, (2) Physiologic Data Source, (3) Harvested data, (4) Data Analysis, (5) Clinical Evaluation and (6) Clinical Implementation. Fig. 2 depicts an overview of the thematic taxonomies.

This theme catalogues domain specific AD research to examine its reusability and portability across other CCUs. The taxonomy developed for this theme is: ICU, PICU, NICU, OR and Other relevant studies. This taxonomy, tabulated in Table  I , reveals that almost all techniques are evaluated in a single domain, with the exception of [23] which is evaluated in the ICU, PICU and OR. In general, trends in physiologic signals display similar dynamics across CCU domains [53] . This drives the hypothesis that AD algorithms could be modified for use across different critical care settings. However, domain specific information such as types and frequency of data, and patient demographics such as range of age, weight and medical condition may be directly, or indirectly, hard coded in the algorithm. Therefore, it is important to consider the original domain of clinical application prior to attempting application elsewhere.

This theme synthesizes the methodologies used to source physiologic data. The theme taxonomy is: Physiologic Monitor, Inclusion/Exclusion Criteria and Sample Size, as tabulated in the second column of Table II . The types, frequency, numeric values and quality of data produced by patient monitors (and probes) differs between models from the same or different OEMs. This is due to different built-in proprietary signal preprocessing, inclusive of AD [28, 54] [55] ; Nellcor OxiMax N-600x (Covidien-Nellcor, Boulder, CO, USA) [55, 56] ; Philips FAST MP50 [56] and Philips Intellivue MP70 (Philips, Germany) [57] . Therefore, knowledge of the monitor OEM and model is important for recognizing different biases introduced in the data. Some studies may collect data under certain inclusion and exclusion criteria as shown in Table II . Sample size is almost always given in the literature. The discretion generally lies with the researcher to determine if enough data is available to draw a statistically and clinically sound decision. Sample size may be deduced based on trial, availability of resources and study requirements.

The taxonomy developed under this theme is: Data Type and Acquisition/Sampling or Storage Frequency. In Table II , the top half of each cell in column three summarizes the types and frequencies of data harvested in each study. Knowledge of the OEM is also a convenient indicator of this information. Routinely harvested or monitored physiologic data types in critical care include: Electrocardiograph (ECG), Heart Rate (HR), Breathing or Respiratory Rate (RR), Impedance Respiratory Wave (IRW), Noninvasive Oxygen Saturation (SpO 2 ), Invasive or noninvasive Arterial Oxygen Saturation (SaO 2 ), Temperature (Temp), the set of Blood Pressure (BP) measurements (namely Systemic Artery, Pulmonary Artery, Central Venous, Systolic (SBP), Diastolic (DBP) And Mean (MBP) Pressures, Arterial (ABP)), Maximal Airway Pressure (MAP), Expired Air Volume (EV), Minute Ventilation (MV) and Transcutaneous Partial Pressures of Carbon Dioxide (TpCO 2 ) and Oxygen (TpO 2 ). Frequency of the data is restricted by analog to digital sampling capability of the monitor [58] . Storage frequency depends on data logging capabilities of both the OEM monitor and the hardware and software mechanism used for storing study data. The synthesis in Table II demonstrates that physiologic data are acquired, derived, sampled and stored at varying frequencies. It is common for AD algorithms to be hard coded to input particular types of data streams having specific frequencies.

High frequency signals, such as continuous waveforms of ECG, ABP and PPG, are sampled at 100 Hz or more. Data that form low frequency time series, such as HR, SBP, DBP and SpO 2 , are either time-averaged at a rate of once every second to once every minute from high frequency signals; or measured intermittently every half hour to an hour such as temperature and non-invasive BP.

Analytic aspects of AD algorithms are reviewed using the following thematic taxonomy: Dimensionality, Focus, Signal Quality and Clinical Contribution. Theme findings are summarised in Table II , in the bottom half of each cell in column three. Dimensionality represents the number of variables or data types that an algorithm is capable of analyzing. According to Imhoff et al. [37] , both the clinical problem and the approach to solving it are (a) univariate when a single feature of a specific data stream is analysed; or (b) multivariate when results are derived from simultaneous analysis of multiple variables; and in between these two approaches lays (c) the univariate clinical problem solved using multivariate data. A new definition for multivariate analysis has emerged in recent research, and as such should be appended to the above as: (d) an algorithm is multivariate if it analyses different metrics derived from the same physiologic signal. Examples of multivariate type (d) research on the PPG is found in [59] and on multi-lead ECG in [34, 50, [60] [61] [62] [63] A uniquely different approach to multivariate analysis is found in [64, 65] , where two different data streams were acquired from the same probe making their physiologic correlation easier to exploit.

The Data Analysis theme characterizes the focus of each algorithm as: (1) stream; (2) patient; and/or (3) disease-centric. A stream-centric algorithm aims to indicate, quantify or improve signal quality of the data stream for increased reliability. Although the term patient-centric has broad implications in health care, it means here that the algorithm was trained on patient specific data and is therefore heavily tailored to each sample patient in the study. For example, baseline data from a particular patient may be required to instantiate an algorithm. BioSign [66] is an example of a realtime, automated, stream and patient-centric system. It produces a single-parameter representation of patient status by fusing five dimensions of vital sign data. A disease-centric methodology focuses on identifying or predicting a specific disease or a clinically significant outcome. The bradycardia detector in [67] and the prognostic tool for Late Onset Neonatal Sepsis (LONS) in [68] are both examples of stream and disease-centric approaches. The clinical contribution taxonomy reveals two unique configurations of AD: (i) Standalone AD and (ii) Coupled AD. As Fig. 1 illustrates, standalone AD techniques typically output filtered or original physiologic data, annotations and SQI. Standalone techniques are labeled as AD under clinical contribution in Table II . Coupled AD techniques are coded as part of algorithms that identify and/or filter artifacts similar to standalone AD techniques with the additional ability for CED or PD. Coupled AD configuration is shown in Fig. 1 and documented under clinical contribution in Table II. Signal Quality is a key element in this taxonomy. Missing segments, error, noise and artifacts inevitably affect data quality thus adversely impacting analytic accuracy and reliability [69] . To address this issue, Clifford et al. [70] recommend that an SQI calibrated to provide a known error rate for a given value of the SQI be made available for each datum. Nizami et al. [71] infer that it suffices for SQIs to be available at a frequency relative to the requirement of another AD, CED or PD application that consume the SQIs. The latter is particularly relevant when down sampled data is required by CED or PDs.

It is hypothesized that the performance of post processing AD algorithms can improve by consuming streaming SQIs output in parallel for each data stream by patient monitors [72] . However, use and delivery of SQIs is not yet standardised across OEM monitors. For example, in case of a lead disconnection the Philips IntelliVue (Philips, Germany) monitor outputs a random value above 8 million in data streams such as the ECG, IRW, HR, RR, SpO 2 and BP. This also generates a corresponding alert type of 'medium priority technicalalarm'loggedatavalueof"2".However,thisvalue is qualitative and not quantitative. The Delta, Gamma, Vista, Kappa and SC6002 -SC9000 monitor series (Dräger Medical Systems, Lubeck, Germany) output an SQI value between 0-100% for the electroencephalography (EEG) channel. This SQI is calculated using sensor impedance data, artifact information and other undisclosed variables. The same monitors also output a fixed label 'ARTF' indicating artifact on any monitored data stream [73, 74] . For example, the monitor classifies QRS complexes only at ECG values > 0.20 mV for widths > 70ms. An artifact condition 'ARTF' may be declared when the ECG signal does not meet these minimum criteria. Although OEM monitors may output signal quality information, there is no logical way to compare the SQIs produced by different monitors. Two confounding reasons are: (1) difference in quantification of SQI; and (2) lack of literature on proprietary algorithms. This reduces researchers' ability to determine how data has been affected from acquisition to logging. As a result, a post processing AD algorithm may need to be strictly matched to input data sourced from a particular OEM monitor as shown in Fig. 1 . Comparative studies led by Masimo [75, 76] declare that its 510 (k) FDA approved RADICAL SET technology has the highest quality measure called Performance Index (PI) as compared to 19 other OEM POs. In these publications, Masimo defines and calculates PI as the percentage of time during which a PO displayed a current SpO 2 value that was within 7% of the simultaneous control value. However, the Masimo SET technology does not automatically evaluate or log this quantitative SQI. Another quality measure called Dropout rate (DR) was calculated in [76] , which equals the percentage of measurement time during which no current SpO 2 values are displayed. Although Masimo SET showed equal or worse DR than two Datex-Ohmeda POs, the reasons for this data loss are not discussed by Masimo. Independent research groups that compared OEM POs in [55] [56] [57] neither researched the effect of the difference in data characteristics on SQIs nor did they mention if the POs output SQIs. Through a recent discourse in [77] on historic developments of the Masimo SET technology, its OEM has replied to Van Der Eijk et al. [56] , claiming greater accuracy in unstable conditions, such as motion artifact and low perfusion, leading to lower false alarm rates. However, [77] does not describe any SQIs that can be consumed meaningfully by other AD, CED or PD applications. This review recommends that AD algorithms that produce SQIs, such as PI and DR amongst others, be evaluated upon data acquired in [55] [56] [57] to contribute towards future AD research.

It follows that AD algorithms designed to post process OEM monitor data must also consume and deliver standardised SQIs. In this way another AD algorithm or a CED or PD mechanism can make informed choices concerning data quality and validity. Review results in Table  II show the increasing trend in SQI development. However, no framework exists to uniformly deliver, compare or combine these SQIs for integration with clinical workflows.

Patient safety requires clinical evaluation of algorithms prior to real-time clinical implementation. This theme reviews clinical evaluation methods bases on this taxonomy: Data Annotation, Mode and Performance Metric. Results are given in Table II , in the top half of each cell of column four. There are no rules that define gold standards for clinical evaluation of AD performance. Each study sets its own gold standard against which its performance is evaluated. This includes evaluations of OEM monitors. Annotated physiologic data, where available, typically serve as the gold standard for validation studies. Events of interest in the data, such as artifacts and clinically significant events, are marked in realtime or retrospectively. The onus of perceiving what constitutesan"event"isontheexpertreviewers,whoidentify the event to the best of their knowledge. Inter-reviewer variability is the significant [78] or subtle [27, 79] difference known to arise when the same dataset is annotated by different reviewers. Retrospective data annotation has been supported by video monitoring in [27, 31, 36] . Video monitoring is only useful when the event is visually perceptible such as sleep movements, certain seizures and routine care. However, it cannot capture crucial physiologic changes such as HR deceleration or BP elevation. The advantage of real-time annotations is recording of richer and more accurate content with input from staff on duty. However, this can be costly and requires cooperation from busy staff. Study data is either collected in real-time from patient monitors or acquired in an "offline"mode for secondary analysis from existing databases. Review results show that majority of AD techniques were validated on offline patient data and very few were tested in real-time CCU environments. Table II also documents the types of performance metrics used in each research. It shows the common trends that will help future researchers to design and compare different algorithms by evaluating them using the same metrics. Numeric comparison of these performance metrics can be found elsewhere in [37, 43, 48, [50] [51] [52] . Performance metrics need to be interpreted very carefully since statistical significance, or absence thereof, is not always representative of clinical significance, or absence thereof. For example, one missed clinical event may not signify a statistical difference in the sensitivity of one OEM monitor over another. However, the same event could be very important clinically and crucial that it not be missed even once. Theoretically, a missed event or a false alarm are caused by artifacts of various types; for example, motion artifact and power line or optical noise induced in an attached or detached sensor. Studies that collected real-time annotated data or video monitored data, such as [27, 31, 36, 80-86] among others, can utilise the same data sets to develop and validate SQIs. Compatible SQIs can be used to compare performance of different AD algorithms and OEM monitors. Performance metrics, for example sensitivity and specificity in alarm studies, can be re-evaluated taking into consideration the SQI at each alarm instance

This theme reviews the clinical implementation status of AD techniques. Theme results are given in Table II , in the bottom half of each cell in column four. This review reveals that the vast majority of AD techniques that are published have not been put into clinical practice. This section critically reviews implementation of some commercialized OEM monitors and the very few techniques developed by independent research groups that made their way into clinical workflows.

The Philips IntelliVue monitoring system (Royal Philips Electronic, Netherlands) features Guardian Early Warning Score (EWS) allowing each hospital to choose its own scoring criteria; Neonatal Event Review which detects apnoea, bradycardia and desaturation; Oxy-cardiorespirography (Oxy-CRG) with compressed trends of a neonate's HR, RR, and SpO 2 ; and ProtocolWatch that is claimed to reduce sepsis mortality rates. Presumably Intellivue preprocesses patient data for artifacts prior to CED or PD, however, no validation studies or algorithm details of this system are published. GE Intellirate TM monitor (Milwaukee, WI, USA) issues asystole, bradycardia and tachycardia alerts by fusing ECG, ABP and PO data. It was evaluated by GE on a small population of 55 CCU patients in 2002 [87] . The evaluation was critiqued in [88] for lack of description of patient demographics and algorithm specifications. GE has republished the exact same study in 2010. The Saphire clinical decision support system [89] uses Intellirate TM technology, but does not evaluate it. Multi-lead ECG arrhythmia detection is deployed by GE in Datex-Ohmeda Bedside Arrhythmia Monitoring (Milwaukee, WI, USA), MARS Ambulatory ECG system and MARS Enterprise (Freiburg, Germany). MARS uses the OEM's EK-Pro Arrhythmia Detection Algorithm which has been evaluated in over 2000 monitored hours spanning at least 100 patients. Surely, these techniques fall under the category of AD coupled with CED and PD. However, literature lacks comparison between different models marketed by the same or different OEMs.

OEM Covidien-Nellcor (Boulder,CO, USA) has developed RR oxi , a coupled AD and PD technology that derives RR from PO. RR oxi has been validated in real-time on 139 healthy subjects in [18] . The OEM is commended for this substantial evaluation. However, healthy subjects are not representative of patient populations which the device is intended to monitor. RR oxi has been validated retrospectively in 12 patients with congestive heart failure, by evaluating 20 minutes of data from each patient [19] . However, larger studies that investigate patient populations with several different pathophysiologies are required to convince clinicians to adopt another patient monitoring technology in their workflows. Fidelity 100 is an FDA 510 (k) approved wireless ECG monitor developed by Signalife (Studio City, CA, USA), which was evaluated in real-time in 54 patients undergoing percutaneous coronary intervention [83] . Although these monitors come with different settings applicable for use in different types of CCUs, validation studies on population data from all application domains are not found. There are a growing number of online open source physiologic databases, such as Capnobase [90, 91] , FDA ECG Warehouse [92] , hemodynamic parameter database [93] , and PhysioNet [94] . It is recommended that these databases be used to compare and validate OEM monitors of different makes and models.

Rest of this section reviews clinical implementation of AD research developed by independent research groups. CIMVA (Therapeutic Monitoring Systems Inc., Ottawa, Canada) is a patented multi-organ variability analytics technology developed by Seely et al. [95] [96] [97] [98] . It is an online tool comprising of multiple coupled AD and CED algorithms with SQIs. Its AD performance is evaluated in [97] . CIMVA research can benefit from the recommendations made in the next section regarding common interfaces and formalised SQIs. This will allow for new AD, CED and PD algorithms to be integrated and tested as part of the CIMVA architecture. Otero et al. have implemented TRACE, a graphical tool which allows clinicians to edit monitoring rules and criteria in realtime. Coupled AD and CED algorithms based on fuzzy set theory input these customized criteria to generate patient alarms in [81, 99] , and detect sleep apnoea in [100] . Given its promising results, evaluation of TRACE against similar OEM monitors is recommended. The research conducted in 1999 by Schoenberg et al. [54] was integrated as part of the commercially available iMDsoft Clinical Information System [101]. However, algorithmic details and evaluation were never published. Artemis is a real-time data analytics system currently undergoing clinical evaluation in multiple NICUs around the globe [102] [103] [104] . As part of the Artemis framework, Nizami et al. [71] present SQI processing to improve the performance of coupled AD and CED algorithms for LONS. Ongoing Artemis research includes AD [80] ; CED of Apnoea of Prematurity [5, 15, 105] ; as well as pain management [9] . Adoption of the structured approach to AD recommended in this review can enhance the clinical performance of coupled AD and CED in Artemis. The coupled AD and CED algorithm for Bradycardia by Portet et al. [67] was evaluated on offline NICU data with the intent of integration with the BabyTalk project. BabyTalk's proof of concept has been described in several publications [106] [107] [108] [109] [110] . However, latest research by Hunter et al. in 2012 [111] infers that a long road lies ahead, including necessary clinical trials, before BabyTalk could be implemented in real-time clinical workflows. Several new AD and CED algorithms can be tested to improve outcomes of this project by incorporating formalised interfaces as recommended in this review. The patented HeRO™ system (Medical Predictive Science Corp., Charlottesville, VA, USA) that scores neonatal Heart Rate Variability (HRV) for predicting LONS is developed using coupled AD and CED algorithms [68] . It conducts a multivariate type (d) analysis of the ECG with multiple coefficients yielding a more sensitive result [84] . The algorithms in this 510 (k) FDA approved device have been extensively described and evaluated both offline and in realtime by Moorman et al. [84, [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] . This pioneering research has shown promising reduction in neonatal mortality by 2% in a randomized control trial on 3003 preterm babies across nine NICUs in the US [114] . The drawback of this trial was a 10% increase in blood work and 5% more days on antibiotics in the monitored infants. Ironically, this constitutes the original problem this research set out to resolve in [119] . It is recommended that other variability measures, such as those of RR as in [10] and PPG as in [123] , be evaluated in comparisonwiththeHeRO™HRVscoretocomeupwith(a) individual scores for each data type; and (b) a composite score that exploits sensor fusion for improved outcomes. BioSign [66] has been evaluated retrospectively and in real-time in a number of clinical studies including randomized control trials in Europe and the US before 2006. However, no later publications could be located.

This section derives conclusions from the thematic review. Post processing AD techniques are highly domain specific. This necessitates modification for validation and reuse in a different CCU domain. Algorithms may be hard coded to input OEM specific data types and frequency. This limits their use with different OEM monitor data. They may be validated under certain inclusion/exclusion criteria which need to be considered when applying the techniques in other contexts. Acquisition and sampling frequency play an important role in patient management since a critically ill patient's condition may deteriorate to a life threatening extent within seconds. Reusability is deterred when such implicit limitations are not expressed. Therefore, adoption of a standardised structured approach for design and reporting of standalone and coupled AD research is recommended. Conformity to generic input and output interfaces will ensure presence of all pertinent information. These interfaces, with common definitions for data type, frequency, length and SQIs, shall allow for matched selection and composition with other AD, CED or PD algorithms. Results from the first three themes are useful in selecting one or more AD algorithms that fulfill data requirements of given CED or PD techniques. Selected AD algorithms can be mixed and matched to discover optimal compositions for varying clinical requirements.

OEM monitors marketed for use across different CCUs have undisclosed built-in preprocessing algorithms, inclusive of AD. Moreover, studies validating their use in different CCU domains and patient population are scarce. The resulting unknown bias imparted in OEM data leads to inevitable variance in analytic results which can effect clinical decisions. This variability can be decreased if monitors output comparable standardised SQIs. As of yet, SQIs do not conform to any standards and are derived differently in each publication, whether it be SQIs delivered by OEM patient monitors or by AD algorithms developed by independent research groups. Interestingly, none of the reviewed algorithms reported using SQIs provided by OEM monitors. Clinical utility of SQIs can be enhanced by using formalised definitions such that SQIs output by different preprocessing and post processing AD algorithms are comparable as well as compatible. An SQI matched to the same set of definitions is also proposed as a requirement at the input of AD, CED or PD algorithms. The objective is to enable the AD, CED or PD algorithm to compare the incoming SQIs generated by one or more AD techniques with their required SQI value. The CED or PD algorithm may then accept or reject incoming data segments based on fulfillment of the required SQI value. In conclusion, standardised SQIs are vital to allow informed clinical choices concerning use and validity of physiologic data.

Results of the Clinical Evaluation theme show that majority of AD techniques are validated on offline data and very few have been evaluated in real-time CCU environments. Clinical Implementation theme reveals that AD techniques developed by independent research groups have rarely found their way into clinical implementation. This leads to the inference that a gap exists between research efforts in AD and their utilization in real-time clinical workflows.

Whereas real-time clinical implementation of AD algorithms is noticeably lacking, there is growing interest amongst clinicians to use CED and PD for automated clinical decision support in CCUs, such as in [8, 104, [124] [125] [126] [127] [128] [129] [130] [131] [132] [133] [134] [135] . Physiologic signal quality assessment through integration of AD can improve the outcome, reliability and accuracy of CED and PD research. The conclusions and recommendations of this review provide new research direction for promoting integration of AD in real-time clinical workflows. Salatian [26] OEM Not specified/ 1 data segment BP at 1 Hz Not discussed/ Offline/ Graphical display Univariate/ Stream-centric/ No SQI/ AD None 

Why should we integrate biomarkers into complex trials?

Review and classification of variability analysis techniques with clinical applications

Why doesn't healthcare embrace simulation and modeling? What would it take

Analysis of continuous oxygen saturation data for accurate representation of retinal exposure to oxygen in the preterm infant

Classifying Neonatal Spells Using Real-Time Temporal Analysis of Physiological Data Streams: Algorithm Development

Test characteristics of an automated age-and temperature-adjusted tachycardia alert in pediatric septic shock

Accuracy of an expanded early warning score for patients in general and trauma surgery wards

A novel technique for identifying patients with ICU needs using hemodynamic features

Identification of Noxious Events For Newborn Infants With a Neural Network

Variability analysis with analytics applied to physiological data streams from the neonatal intensive care unit

A process mining driven framework for clinical guideline improvement in critical care

Pulse oximetry saturation patterns detect repetitive reductions in airflow

Implementation of a real-time computerized sepsis alert in nonintensive care unit patients

Using modified multivariate bag-of-words models to classify physiological data

A Framework for Multidimensional Real-Time Data Analysis: A Case Study for the Detection of Apnoea of Prematurity

Online apnea-bradycardia detection using hidden semi-Markov models

Deriving respiration from photoplethysmographic pulse width

Developing an algorithm for pulse oximetry derived respiratory rate (RR oxi): A healthy volunteer study

Accuracy of Continuous Noninvasive Respiratory Rate Derived From Pulse Oximetry in Congestive Heart Failure Patients

Artificial arterial blood pressure artifact models and an evaluation of a robust blood pressure and heart rate estimator

Robust heart rate estimation from multiple asynchronous noisy sources using signal quality indices and a Kalman filter

A robust sensor fusion method for heart rate estimation

Robust sensor fusion improves heart rate estimation: Clinical evaluation

Instrumentation and practice standards for electrocardiographic monitoring in special care units. A report for health professionals by a task force of the council on clinical cardiology

Anaesthesia monitor alarms: a theory-driven approach

Filtering of ICU monitor data to reduce false alarms and enhance clinical decision support

Collection of annotated data in a clinical validation study for alarm algorithms in intensive care-a methodologic framework

The Problem of Artifacts in Patient Monitor Data During Surgery: A Clinical and Methodological Review

Signal quality measures for pulse oximetry through waveform morphology analysis

Patterns of unexpected in-hospital deaths: A root cause analysis

Electroencephalography (EEG) recording techniques and artefact detection in early premature babies

Common electrocardiographic artifacts mimicking arrhythmias in ambulatory monitoring

Artifactual electrocardiographic change mimicking clinical abnormality on the ECG

Signal quality indices and data fusion for determining clinical acceptability of electrocardiograms

Reducing hospital noise: A review of medical device alarm management

The wolf is crying in the operating room: Patient monitor and anesthesia workstation alarming patterns during cardiac surgery

Smart alarms from medical devices in the OR and ICU

Users' opinions on intensive care unit alarms -A survey of German intensive care units

Managing alarm fatigue in cardiac care

Monitor alarm fatigue: An integrative review

Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit

Automatic detection of ECG electrode misplacement: a tale of two algorithms

Signal quality in cardiorespiratory monitoring

Electrocardiogram signal quality measures for unsupervised telehealth environments

Applications of supervised learning to biological signals: ECG signal quality and systemic vascular resistance

Improving the quality of ECGs collected using mobile phones: The PhysioNet/Computing in Cardiology Challenge

Medical device alarms

Robust filters for intensive care monitoring: Beyond the running median

Alarms in the intensive care unit: How can the number of false alarms be reduced?

Threshold-based system for noise detection in multilead ECG recordings

Real-time signal processing by adaptive repeated median filters

Alarm algorithms in critical monitoring

A Cusum-Based Multilevel Alerting Method for Physiological Monitoring

Making ICU alarms meaningful: a comparison of traditional vs. trend-based algorithms

Oxygen targeting in preterm infants using the Masimo SET Radical pulse oximeter

New-generation" pulse oximeters in extremely low-birth-weight infants: How do they perform in clinical practice?

Letters: Comparing oxygen targeting in preterm infants between the Masimo and Philips pulse oximeters

Online pattern recognition based on a generalized hidden Markov model for intraoperative vital sign monitoring

Dynamic time warping and machine learning for signal quality assessment of pulsatile signals

Reducing False Intracranial Pressure Alarms using Morphological Waveform Features

QRS detection based ECG quality assessment

Evaluation of an algorithm based on single-condition decision rules for binary classification of 12-lead ambulatory ECG recording quality

Automatic ECG quality scoring methodology: mimicking human annotators

Clinical diagnosis of pneumothorax is late: Use of trend data and decision support might allow preclinical detection

Artifact Detection in the PO2 and PCO2 Time Series Monitoring Data from Preterm Infants

Integrated monitoring and analysis for early warning of patient deterioration

Evaluation of On-Line Bradycardia Boundary Detectors from Neonatal Clinical Data

HeRO Technical Publications

Decision tool for the early diagnosis of trauma patient hypovolemia

Robust parameter extraction for decision support using multimodal intensive care data

Service oriented architecture to support real-time implementation of artifact detection in critical care monitoring

On-line novelty detection for artefact identification in automatic anaesthesia record keeping

Instructions for use: Infinity Delta Series

Infinity gateway VF5.0 protocol handbook

Performance of three new-generation pulse oximeters during motion and low perfusion in volunteers

Motion-resistant" pulse oximetry: A comparison of new and old models

Reply to "'new-generation' pulse oximeters in extremely low-birth-weight infants

Individual and Joint Expert Judgments as Reference Standards in Artifact Detection

The practical management of artifact in computerised physiological data

On the integration of an artifact system and a real-time healthcare analytics system

Addressing the flaws of current critical alarms: a fuzzy constraint satisfaction approach

Classifying alarms in intensive care -Analogy to hypothesis testing

Evaluation of Novel ECG Signal Processing on Quantification of Transient Ischemia and Baseline Wander Suppression

Heart rate characteristics monitoring for neonatal sepsis

An annotated data collection system to support intelligent analysis of Intensive Care Unit data

Crying wolf: False alarms in a pediatric intensive care unit

Executive Summary: Intellirate Technology for Patient Monitoring

Reducing false alarm rates for critical arrhythmias using the arterial blood pressure waveform

IST-27074 SAPHIRE: Intelligent healthcare monitoring based on a semantic interoperability platform

Photoplethysmogram signal quality estimation using repeated Gaussian filters and cross-correlation

Respiratory rate estimation using respiratory sinus arrhythmia from photoplethysmography

The cardiac safety research consortium ECG database

Thedatabase of the cardiovascular system related signals

PhysioBank, PhysioToolkit, and PhysioNet : Components of a New Research Resource for Complex Physiologic Signals

Continuous multiorgan variability monitoring in critically ill patients -complexity science at the bedside

Annual International Conference of the IEEE

Monitoring and Identification of Sepsis Development through a Composite Measure of Heart Rate Variability

Feasibility of continuous multiorgan variability analysis in the intensive care unit

Continuous Multi-Parameter Heart Rate Variability Analysis Heralds Onset of Sepsis in Adults

Intelligent alarms for patient supervision

A structural knowledgebased proposal for the identification and characterization of apnoea episodes

Trends and opportunities for integrated real time neonatal clinical decision support

Next Generation Neonatal Health Informatics with Artemis

Real-time analysis for intensive care: Development and deployment of the artemis analytic system

A framework to model and translate clinical rules to support complex real-time analysis of physiological and clinical data

Using Temporal Constraints to Integrate Signal Analysis and Domain Knowledge in Medical Event Detection

Automatic generation of textual summaries from neonatal intensive care data

Text content and task performance in the evaluation of a Natural Language Generation system

From data to text in the neonatal intensive care Unit: Using NLG technology for decision support and information management

Summarising complex ICU data in natural language

Automatic generation of natural language nursing shift summaries in neonatal intensive care: BT-Nurse

Predictive monitoring for early detection of subacute potentially catastrophic illnesses in critical care

Cardiovascular oscillations at the bedside: early diagnosis of neonatal sepsis using heart rate characteristics monitoring

Mortality reduction by heart rate characteristic monitoring in very low birth weight neonates: A randomized trial

Nearest-Neighbor and Logistic Regression Analyses of Clinical and Heart Rate Characteristics in the Early Diagnosis of Neonatal Sepsis

Heart rate characteristics and clinical signs in neonatal sepsis

Heart rate characteristics: Novel physiomarkers to predict neonatal infection and death

Sample Asymmetry Analysis of Heart Rate Characteristics with Application to Neonatal Sepsis and Systemic Inflammatory Response Syndrome

Abnormal heart rate characteristics preceding neonatal sepsis and sepsis-like illness

Sample entropy analysis of neonatal heart rate variability

Toward the early diagnosis of neonatal sepsis and sepsis-like illness using novel heart rate analysis

The Dynamic Range of Neonatal Heart Rate Variability

Peripheral photoplethysmography variability analysis of sepsis patients

A pattern-based approach for representing condition-action clinical rules into DSSs

Photoplethysmographic derivation of respiratory rate: A review of relevant physiology

COSARA: Integrated service platform for infection surveillance and antibiotic management in the ICU

KETO: A knowledge editing tool for encoding condition -Action guidelines into clinical DSSs

Clinical knowledge-based inference model for early detection of acute lung injury

Detection of QT prolongation using a novel electrocardiographic analysis algorithm applying intelligent automation: Prospective blinded evaluation using the Cardiac Safety Research Consortium electrocardiographic database

Impact of sedation and organ failure on continuous heart and respiratory rate variability monitoring in critically ill patients: A pilot study

Informatics infrastructure for syndrome surveillance, decision support, reporting, and modeling of critical illness

Implementation and evaluation of electronic clinical decision support for compliance with pneumonia and heart failure quality indicators

An architecture for online comparison and validation of processing methods and computerized guidelines in intensive care units

A modular clinical decision support system clinical prototype extensible into multiple clinical settings

Assemblies of Heterogeneous Technologies at the Neonatal Intensive Care Unit

Predictive combinations of monitor alarms preceding in-hospital code blue events

Robust heart beat detection from photoplethysmography interlaced with motion artifacts based on empirical mode decomposition

Reducing false alarms of intensive care online-monitoring systems: An evaluation of two signal extraction algorithms

On-line adaptive trend extraction of multiple physiological signals for alarm filtering in intensive care units

On-line segmentation algorithm for continuously monitored data in intensive care units

Specificity improvement for network distributed physiologic alarms based on a simple deterministic reactive intelligent agent in the critical care environment

Reduction of false arterial blood pressure alarms using signal quality assessement and relationships between the electrocardiogram and arterial blood pressure

Detection of artifacts in monitored trends in intensive care

Patient-specific learning in real time for adaptive monitoring in critical care

Event discovery in medical time-series data

Poor prognosis for existing monitors in the intensive care unit

Robust classification of neonatal apnoea-related desaturations

A new algorithm for detecting central apnea in neonates

Automatic detection of apnoea of prematurity

Factorial Switching Linear Dynamical Systems Applied to Physiological Condition Monitoring

Building ICU artifact detection models with more data in less time

Multiple signal integration by decision tree induction to detect artifacts in the neonatal intensive care unit

On the road to predictive smart alarms based on a networked operating room

Sensor Fusion Using a Hybrid Median Filter for Artifact Removal in Intraoperative Heart Rate Monitoring

An evaluation of a novel software tool for detecting changes in physiological monitoring

Real-time pulse oximetry artifact annotation on computerized anaesthetic records

Physiologic trend detection and artifact rejection: a parallel implementation of a multi-state Kalman filtering algorithm

Detection of false alarms using an integrated anesthesia monitor

The association between vital signs and major hemorrhagic injury is significantly improved after controlling for sources of measurement variability

Improvement of ECG signal quality measurement using correlation and diversity-based approaches

Signal quality estimation with multichannel adaptive filtering in intensive care settings

Information Technology in Biomedicine

Computer algorithms for evaluating the quality of ECGs in real time

CinC challenge -Assessing the usability of ECG by ensemble decision trees

Data driven approach to ECG signal quality assessment using multistep SVM classification

Automated detection of sleep apnea from electrocardiogram signals using nonlinear parameters

Modeling Respiratory Movement Signals During Central and Obstructive Sleep Apnea Events Using Electrocardiogram

Heart disease classification through HRV analysis using Parallel Cascade Identification and Fast Orthogonal Search

Data fusion for improved respiration rate estimation

Multivariate Analysis of Blood Oxygen Saturation Recordings in Obstructive Sleep Apnea Diagnosis

Detection of decreases in the amplitude fluctuation of pulse photoplethysmography signal as indication of obstructive sleep apnea syndrome in children

Detection of obstructive sleep apnea in children using decreases in amplitude fluctuations of PPG signal and HRV

A Method for Automatic Identification of Reliable Heart Rates Calculated from ECG and PPG Waveforms

TRACE, Tool foR anAlyzing and disCovering pattErns