key: cord-0027382-2dcunaw0
authors: Horton, William B.; Barros, Andrew J.; Andris, Robert T.; Clark, Matthew T.; Moorman, J. Randall
title: Pathophysiologic Signature of Impending ICU Hypoglycemia in Bedside Monitoring and Electronic Health Record Data: Model Development and External Validation
date: 2021-06-24
journal: Crit Care Med
DOI: 10.1097/ccm.0000000000005171
sha: 91d2bb7eef4761a445a209d6ef14b7647b90bb79
doc_id: 27382
cord_uid: 2dcunaw0

We tested the hypothesis that routine monitoring data could describe a detailed and distinct pathophysiologic phenotype of impending hypoglycemia in adult ICU patients. DESIGN: Retrospective analysis leading to model development and validation. SETTING: All ICU admissions wherein patients received insulin therapy during a 4-year period at the University of Virginia Medical Center. Each ICU was equipped with continuous physiologic monitoring systems whose signals were archived in an electronic data warehouse along with the entire medical record. PATIENTS: Eleven thousand eight hundred forty-seven ICU patient admissions. INTERVENTIONS: The primary outcome was hypoglycemia, defined as any episode of blood glucose less than 70 mg/dL where 50% dextrose injection was administered within 1 hour. We used 61 physiologic markers (including vital signs, laboratory values, demographics, and continuous cardiorespiratory monitoring variables) to inform the model. MEASUREMENTS AND MAIN RESULTS: Our dataset consisted of 11,847 ICU patient admissions, 721 (6.1%) of which had one or more hypoglycemic episodes. Multivariable logistic regression analysis revealed a pathophysiologic signature of 41 independent variables that best characterized ICU hypoglycemia. The final model had a cross-validated area under the receiver operating characteristic curve of 0.83 (95% CI, 0.78–0.87) for prediction of impending ICU hypoglycemia. We externally validated the model in the Medical Information Mart for Intensive Care III critical care dataset, where it also demonstrated good performance with an area under the receiver operating characteristic curve of 0.79 (95% CI, 0.77–0.81). CONCLUSIONS: We used data from a large number of critically ill inpatients to develop and externally validate a predictive model of impending ICU hypoglycemia. Future steps include incorporating this model into a clinical decision support system and testing its effects in a multicenter randomized controlled clinical trial.

Beyond poor clinical outcomes, inpatient hypoglycemia also carries financial implications. A recent study of 43,659 admissions within the Florida Hospital System found that even one episode of hypoglycemia resulted in a total cost of care that was $10,405 greater than that in patients whose blood glucose remained within the normal range (8) . With these factors in mind, The Centers for Medicare and Medicaid Services (CMS) has identified inpatient hypoglycemia as a high-priority measurement area and is currently adapting a hypoglycemia prevention measure (National Quality Forum #2363: Glycemic Control-Hypoglycemia) for possible future CMS use. In practice, this measure would incentivize hospitals to implement clinical workflows that facilitate evidence-based glycemic management strategies to reduce the likelihood of hypoglycemic events.

The prevalence of inpatient hypoglycemia is nearly threefold higher in the ICU than that in non-ICU settings (9, 10) , and multiple studies confirm that ICU hypoglycemia is linked to increased morbidity and mortality (6, (11) (12) (13) . Given the strong association between ICU hypoglycemia and poor outcomes, a proactive approach using targeted predictive analytics is needed (14) . One such approach is to retrospectively analyze historical clinical data and develop a prediction tool that determines the individualized risk of ICU hypoglycemia. The possibility of developing such a prediction tool lies in the growing availability of rich clinical datasets stored in a hospital's electronic health record (EHR) system (15) . With the well-established biochemical, hemodynamic, and electrophysiology changes that occur during hypoglycemia (16) , EHRs provide an invaluable resource for prediction tool development. Despite recent advancements in EHRs and machine learning, few studies have focused on model development solely for ICU hypoglycemia (17) . In this study, we used machine learning to test the hypothesis that routine monitoring data could describe a detailed and distinct pathophysiologic phenotype of impending hypoglycemia in adult ICU patients.

We retrospectively analyzed all ICU admissions from October 2013 to August 2017 at the University of Virginia (UVa) Medical Center wherein patients were greater than or equal to 18 years old and received insulin therapy. This included medical (28 beds), surgical-trauma (15 beds), thoracic-cardiovascular postoperative (19 beds), coronary care (10 beds), and neuroscience (12 beds) ICUs. Each ICU was equipped with continuous physiologic monitoring systems whose signals were archived in an electronic data warehouse along with the entire medical record. Monitoring data archival was interrupted in the coronary care and thoracic-cardiovascular postoperative ICUs in July 2015 due to changes in biomedical engineering infrastructure. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines (18) and followed the recommendations set forth by Leisman et al (19) to analyze and report this study (20) . A completed TRIPOD checklist is included in the Supplemental Material (http://links.lww.com/CCM/G587). All study protocols were approved by the UVa Institutional Review Board for Health Sciences Research (protocol number 22152).

The primary outcome was hypoglycemia, defined as any episode of blood glucose less than 70 mg/dL, where 50% dextrose injection (i.e., D50) was also administered within 1 hour. This specific definition was chosen because our EHR hypoglycemia order set includes administration of D50 whenever a blood glucose less than 70 mg/dL is recorded. Secondary outcomes included mortality and length of stay. We focused on physiologic data starting 12 hours before the hypoglycemic episode. As controls, we included data from greater than 12 hours before the hypoglycemic episode and from insulin-treated ICU patients who did not experience hypoglycemia during admission. We censored all data after each initial hypoglycemic episode (i.e., recurrent hypoglycemia was not captured).

We performed modeling in R 4.0.2 (R Core Team 2020, Vienna, Austria) using the "rms" package (21) . For the univariable analysis, we plotted predictiveness curves to show the individual association of 61 vital signs, laboratory values, demographics, and continuous cardiorespiratory monitoring variables with episodes of hypoglycemia compared with no hypoglycemia. We randomly sampled one measurement within 24 hours before to 15 hours after each episode. We calculated the relative risk of hypoglycemia at each decile of the sampled variable and then interpolated the risk to 20 points evenly spaced in the variable range. We repeated this process of sampling, calculating relative risk, and interpolating 30 times. Finally, we averaged the 30 risk estimate curves to obtain a predictiveness curve at the 20 evenly spaced points and displayed results as a heat map.

For multivariable modeling (both for the aggregate ICU model and the individual ICU models), we assessed 41 physiologic variables that were clinically relevant to hypoglycemia and at least 70% available (i.e., a given variable is available for greater than or equal to 70% of the time points for the entire cohort). We used multivariable logistic regression adjusted for repeated measures to relate physiologic data to the hypoglycemia outcome on the entire cohort (21) . We systematically built the model by: 1) removing, blinded to the outcome, the most predictable features correlated more than R 2 of 0.9 with other features, 2) imputing missing values with median values for the study population, 3) building a model with all remaining features and restricted cubic splines (three knots) on each feature with enough unique values (21, 22) , adjusting for repeated measures using the Huber-White method (21) , and 4) using ridge regression (23) to penalize model coefficients, shrinking the effective degrees of freedom to maximize the corrected Akaike information criterion (24, 25) .

We then performed internal validation using 10-fold cross-validation (TRIPOD type 1b model study) (20, 26) . We randomly split the patient admissions into 10 groups, excluded a single group's data as a test set, trained a model on the remaining data using the same features and penalty found above, used that model to estimate the risk of hypoglycemia for the test set, and then calculated the area under the receiver operating characteristic curve (AUROC) in the test set. We repeated this procedure until each of the 10 groups had served as a test set and used the 10 resulting AUROC measurements to estimate the mean and 95% CIs. Although this method calculated out-of-sample predictions with the same features, we made the predictions with slightly different models, one for each test set. We note that predictions were made every hour, there were multiple predictions per patient, and not every patient experienced the primary outcome.

We performed external validation in the Medical Information Mart for Intensive Care (MIMIC-III) (27, 28) . The MIMIC-III database was approved by the Institutional Review Boards of the Massachusetts Institute of Technology (Cambridge, MA) and Beth Israel Deaconess Medical Center. We note that the MIMIC-III Waveform Database Matched Subset consists of all waveform and numeric recordings for which the corresponding clinical record is also available and that this dataset was not limited to insulin-treated patients only.

We analyzed data obtained from 11,847 UVa ICU patient admissions, 721 (6.1%) of whom had one or more hypoglycemic episodes. Table 1 demonstrates baseline demographics, admission characteristics, mortality, and length of stay for our study population. Mortality was about three-fold higher (28.3% vs 9.8%; p < 0.001) and length of stay doubled (15 vs 7 d; p < 0.001) in patients who experienced hypoglycemia (Table 1) . Notably, hypoglycemia was associated with increased mortality even after accounting for age, comorbidities, illness severity, and clinical presentation (p < 0.0001). Black inpatients were overrepresented in the hypoglycemia cohort, comprising 23.4% of that group but only 16.1% overall. Conversely, White inpatients were underrepresented in the hypoglycemia cohort, comprising 74.3% of that group but 81.3% overall.

For external validation in MIMIC-III, we analyzed data obtained from 9,878 ICU patient admissions. Four hundred ninety-three (5.0%) of these ICU patient admissions experienced one or more hypoglycemic episodes. As in the UVa dataset, those who experienced hypoglycemia had higher mortality (27.6% vs 10.7%; p < 0.001) and longer length of stay (13.5 vs 7.1 d; p < 0.001) when compared with those who did not experience hypoglycemia.

Univariable analysis of 61 physiologic and biochemical variables identified trends in each that were associated with hypoglycemia ( Fig. 1) . Several of these variables had nonlinear associations with ICU hypoglycemia (e.g., WBC count and serum potassium), indicating that hypoglycemia risk increased at both the lowest and highest percentiles of these variables. Another notable finding was that serum anion gap demonstrated a strongly positive association with ICU hypoglycemia. We initially attributed this to diabetic ketoacidosis but noted that lactic acid also demonstrated a strongly positive relationship with hypoglycemia risk. These results suggest that the positive relationship between ICU hypoglycemia risk and higher anion gap is likely indicative of severe or worsening illness due to factors (e.g., lactic acidosis and uremia) beyond diabetic ketoacidosis alone.

Multivariable logistic regression modeling identified a signature of 41 independent predictors that best characterized impending ICU hypoglycemia. The top 10 features in decreasing strength of association were serum glucose, serum anion gap, body temperature, serum potassium, serum creatinine, prothrombin time, blood urea nitrogen/creatinine, serum carbon dioxide, the sd of oxygen saturation by pulse oximetry, and serum calcium. These predictors are consistent with prior reports demonstrating that: 1) hypothermia forms a basic aspect of the response to hypoglycemia (29) (30) (31) , 2) adrenaline release and excess insulin (as seen with hypoglycemia) stimulate potassium uptake from the bloodstream (32), and 3) renal dysfunction increases risk for hypoglycemia (33) .

The cross-validated AUROC for our composite ICU hypoglycemia model (including all variables) was 0.83 

Critical Care Medicine www.ccmjournal.org e225 Figure 1 . Heat map depiction of the univariable risk of ICU hypoglycemia as a function of 61 measured physiologic and biochemical variables. Each tile plots the value of the variable on the x-axis against the relative risk of ICU hypoglycemia on the right y-axis. Variables on the left y-axis represent model outputs, with those in red text indicating laboratory values, those in blue text indicating hemodynamic monitoring variables, and those in green text indicating electrophysiology variables. The relative risk color bar ranges from 0.50 to 2.0. A red color saturation indicates higher relative risk of hypoglycemia; a blue color saturation indicates a lower relative risk. <RRI> = mean R-R interval, AF = probability of atrial fibrillation, AGAP = anion gap, Alb = albumin, ALP = alkaline phosphatize, ALT = alanine aminotransferase, AST = aspartate aminotransferase, BE = base excess from arterial blood gas, Bili = bilirubin, BUN = blood urea nitrogen, BUN/Cr = blood urea nitrogen/creatinine, Ca = calcium, Cl = chloride, CO 2 = carbon dioxide, COSEn = coefficient of sample entropy of R-R interval, Cr = creatinine, DBP (cuff) = diastolic blood pressure by cuff measurement, DBP = invasive diastolic blood pressure (mm Hg), DFA = detrended fluctuation analysis applied to R-R intervals, EDR = electrocardiogram-derived respiratory rate (breaths/min), Gluc = glucose, Hco 3 = bicarbonate, Hct = hematocrit, HR = heart rate measured by cardiac telemetry (beats/min), HRV = sd of heart rate by cardiac telemetry (beats/min), HRxEDR = cross-correlation coefficient of heart rate and electrocardiogram-derived respiratory rate, HRxRR = cross-correlation coefficient of heart rate measured by cardiac telemetry and respiratory rate measured by chest impedance, HRxSO2 = the cross-correlation coefficient of heart rate and oxygen saturation, K = potassium, Lact = lactate, LDd = local dynamics density of heart rate, MAP = mean arterial pressure by cuff measurement, MBP = invasive mean blood pressure (mm Hg), Mg = magnesium, Na = sodium, Neut % = neutrophil percentage (%), O 2 = oxygen saturation from arterial blood gas, O2V = the sd of oxygen saturation by pulse oximetry, pH = pH from arterial blood gas, Plt = platelet count, Po 2 = Po 2 from arterial blood gas, PO 4 = phosphorous, PT/INR = prothrombrin time/international normalized ratio, PTT = partial thromboplastin time, Pulse = heart rate measured by pulse oximetry (beats/min), Resp = respiratory rate measured by pulse oximetry, RR = respiratory rate measured by chest impedance, RRV = sd of respiratory rate by chest impedance (breaths/min), RRxSO2 = cross-correlation coefficient of respiratory rate measured by chest impedance and oxygen saturation measured by pulse oximetry, SBP (cuff) = systolic blood pressure by cuff measurement, SBP = invasive systolic blood pressure (mm Hg), SO 2 = oxygen saturation measured by pulse oximetry (%), Spo 2 = cliniciandocumented oxygen saturation (%), sRRI = the sd of R-R intervals, Temp = temperature (°C), TP = total protein, Trop I = troponin I (95% CI, 0.79-0.88) ( Fig. 2A) . To assess the inputs required for acceptable predictive validity, we examined models consisting of laboratory values only (excluding glucose), vital signs only, and continuous monitoring variables only. The respective cross-validated AUROCs for these models were 0.69, 0.67, and 0.65, confirming that each stream can predict hypoglycemia individually. As expected, the composite model performed better (34) . The area under the precision-recall curve (AUPRC) for our composite ICU hypoglycemia model was 0.084 (event rate = 0.0045 and ratio = 20.7). For reference, the AUPRC of a random predictor is the event rate and is therefore necessarily higher for more frequent events (35) . The AUPRC of our predictor is 20.7 times better than a random predictor at this event rate. Although AUROC and AUPRC have the advantage of being threshold-independent, we also evaluated thresholddependent metrics (e.g., sensitivity and positive predictive value). For this, we defined an "alert" by a rise in predicted risk. In this way, we can set the number of alerts by defining the rise in risk required. We found that sending one alert www.ccmjournal.org

March 2022 • Volume 50 • Number 3 per day yields a sensitivity ~24% and positive predictive value ~26%. We also examined performance of prior models developed by our group for prediction of ICU sepsis, intubation, and hemorrhage (22) to determine their predictive capability for ICU hypoglycemia. These models all demonstrated poor predictive ability for ICU hypoglycemia (Fig. 2B) . For example, a model for sepsis in the medical ICU had an AUROC of 0.62 for detection of ICU hypoglycemia. This suggests that our ICU hypoglycemia model is specific to hypoglycemia and not just worsening clinical status.

Our model demonstrated good performance (AUROC: 0.79 [95% CI, 0.77-0.81], and AUPRC: 0.09 [event rate = 0.0082 and ratio = 11.0]) with external validation testing in the MIMIC-III Waveform Database Matched Subset. However, some limitations of the MIMIC-III dataset should be noted: 1) bedside monitoring blood pressure values were significantly lower in some MIMIC ICUs, 2) bedside monitoring vital signs in MIMIC-III were often sampled every 1 minute instead of every 1 second, so the sd differs from our UVa dataset, 3) time stamps for labs in MIMIC-III were blood draw time and not result time, and 4) medication administration was not available, and thus, we could not restrict analysis to insulintreated patients.

The plotted calibration curve for the aggregate ICU hypoglycemia model is shown in Figure 3A . The model demonstrated reasonable calibration within both the UVa and MIMIC-III datasets, with predicted risk rising as relative risk increased. Notably, in both datasets, patients with the lowest 80% of predicted risk had less than average observed risk. Figure 3B demonstrates average risk in relation to timing of hypoglycemic events. The model identified rising hypoglycemia risk ~4-6 hours prior to the hypoglycemic event in both the UVa and MIMIC-III datasets, reflecting a rising degree of physiologic and biochemical abnormality in the hours prior to clinical recognition of hypoglycemia.

We used a "Big Data" analytic approach and applied multivariable logistic regression to describe signatures of ICU hypoglycemia from readily available physiologic and biochemical data collected from the EHR of a large university hospital. Mortality and length of stay were significantly higher in patients who experienced hypoglycemia, and these associations held true after adjusting for variables like age, comorbidities, and illness severity. We also identified racial differences in frequency of hypoglycemia and thought that mechanisms for this are not obvious. Further targeted research that more accurately captures the social construct of race along with appropriate explanatory (though intertwined) biological and sociologic variables is required (36, 37) .

The pathophysiological signature of impending ICU hypoglycemia was composed of 41 different variables and demonstrated good discriminatory capability and reasonable calibration (38) . To our knowledge, this is the first study that incorporates hemodynamic and electrophysiology bedside monitoring data to provide a comprehensive predictive model of ICU hypoglycemia. The features of our model described a pathophysiologic signature that was consistent across different adult ICUs, independent of recent blood glucose trends, and had general similarities to other illnesses (e.g., hypoglycemia, hypothermia, increasing anion gap, and hypocalcemia are metabolic and hemodynamic derangements frequently seen in critically ill states [11, [39] [40] [41] [42] ) but was sufficiently different to warrant its own model. The model identified rising hypoglycemia risk ~4-6 hours prior to the hypoglycemic event in both the UVa and MIMIC-III datasets, suggesting that there is a reasonable timeframe for early intervention prior to occurrence of a hypoglycemic event.

Machine learning has been increasingly used to develop predictive models for inpatient hypoglycemia (15, 17, (43) (44) (45) (46) . Only one report thus far focused on predicting solely ICU hypoglycemia, and this study used classification tree learning for model development (17) . Several other models used logistic or multivariable regression techniques for prediction of inpatient hypoglycemia, but these studies examined only inpatients with a diagnosis of diabetes mellitus (15, 45) , only inpatients who experienced severe hypoglycemia (43, 44) , and only noncritically ill inpatients (46, 47) . Ruan et al (15) recently used average values from the entire admission (based on data available after the admission ended) to compare the ability of advanced machine learning and logistic regression models to retrospectively estimate the risk of hypoglycemia in inpatients with diabetes. The model we present, by contrast, is appropriate for risk prediction at any point during the ICU stay based only on data available at that time. Similar to the present study, Mathioudakis et al (47) recently developed and validated a machine learning model to predict near-term risk of iatrogenic hypoglycemia in hospitalized patients. Their model, however, was trained on and specifically developed for non-ICU admissions and did not exclude blood glucose as a predictor. Our results show that an aggregate ICU hypoglycemia model (including blood glucose, biochemical, and electrophysiology monitoring data) demonstrated significantly higher AUROC values at every detection window when compared with models based on either blood glucose alone or hemodynamic/electrophysiology monitoring data alone (Supplemental to drive blood glucose toward a predetermined target range (48) . These tools improved glycemic control (49, 50) but have limitations: 1) they are not employed in all ICUs, 2) hypoglycemia still occurs in patients being managed with them (49, 51) , and 3) their algorithms do not incorporate many of the pertinent hemodynamic and electrophysiology predictors identified in the current study. Our findings advance the work of other groups and demonstrate that hemodynamic and electrophysiology data augment biochemical data to improve predictive models for ICU hypoglycemia. One possible clinical application of these findings is to present a single calibrated input using weighted nonglucose, noninsulin, and noncarbohydrate predictors to add dimensionality to existing schemes for titrations of antihyperglycemic therapy and carbohydrate exposure.

Appropriate glycemic control is a necessary component of quality-driven inpatient healthcare. In critically ill inpatients, intensive glycemic control reduces hyperglycemia but often leads to subsequent hypoglycemia (11) . The Normoglycemia in Intensive Care Evaluation-Survival Using Glucose Algorithm Regulation trial found that intensive insulin therapy increased 90-day mortality compared with conventional treatment in ICU patients (52) . In that trial, the frequency of severe hypoglycemia was significantly higher with intensive insulin therapy compared with conventional treatment. Other work has shown that even mild hypoglycemia is strongly associated with increased ICU length of stay (13) . Our study found similar associations among ICU hypoglycemia, mortality, and length of stay. We cannot prove causality for these associations, and it may be that hypoglycemia itself is a clinical sign of worsening or severe illness. However, one recent study found that reducing the frequency of inpatient hypoglycemia concomitantly reduced inpatient and 30-day mortality rates (53) . Further trials are needed to determine if direct reduction of ICU hypoglycemia improves clinical outcomes. Toward this goal, the next step for this project is to determine if our prediction model offers clinical impact. We are currently running the model "in the background" of real-time ICU admissions to determine if it prospectively predicts hypoglycemia and assess what clinical events might be prevented with earlier intervention. Future steps include incorporating our model into a clinical decision support system and evaluating its effects on clinical outcomes in a multicenter randomized controlled clinical trial.

Our study has several strengths that should be noted, including the large dataset used for model development. Another strength is the model's ability to immediately quantify the change in hypoglycemia risk from small changes in any of its physiologic variables and then produce a new and continuously updated estimate of ICU hypoglycemia risk in a given patient. The use of variables that are easily accessible from EHR and bedside monitoring data will also allow integration into a clinical decision support system that suggests appropriate interventions based on individual risk levels, ultimately providing a personalized approach to ICU hypoglycemia.

There are also limitations of this study which warrant discussion. First, our model was generated using single-center, retrospective, observational data. Second, our EHR dataset does not quantify status of hypoglycemia awareness, continuous blood glucose monitoring values, or blood glucose self-monitoring values prior to admission. These data may be an important factor in developing inpatient hypoglycemia, though others have pointed out that such data may not be directly applicable to critically ill patients (15) . Finally, we did not assess nutritional intake or medications as predictor variables, though they would without question add information. Our goal was to seek a pathophysiological signature of subclinical hypoglycemia in ICU patients that was not dependent on factors such as practice patterns that can vary significantly among health systems.

In summary, we used data from a large number of critically ill adult patients to test the hypothesis that routine monitoring variables could provide a distinct pathophysiologic phenotype of ICU hypoglycemia. This physiologic signature could provide a basis for future predictive modeling by improving recognition of impending ICU hypoglycemia, informing the design of earlier interventions and measuring their effectiveness, and identifying opportunities for the development of novel therapeutics.

Hypoglycemia prevention in hospital patients: A quality improvement project to prevent severe and recurrent hypoglycemia

Management of Diabetes and Hyperglycemia in Hospitalized Patients

Death during intensive glycemic therapy of diabetes: Mechanisms and implications

Association of hypoglycemia and cardiac ischemia: A study based on continuous monitoring

Hypoglycemia and cardiovascular risks

Hypoglycemia and outcome in critically ill patients

Mortality among hospitalized patients with hypoglycemia: Insulin related and noninsulin related

At a Cost of $10,405 Per Patient Stay, Hypoglycemia in the Hospital Cannot Be Ignored

Inpatient glucose control: A glycemic survey of 126 U.S. hospitals

Understanding hypoglycemia in hospitalized patients

NICE-SUGAR Study Investigators: Hypoglycemia and risk of death in critically ill patients

Mild hypoglycemia is independently associated with increased mortality in the critically ill

Mild hypoglycemia is strongly associated with increased intensive care unit length of stay

Coronavirus disease 2019 calls for predictive analytics monitoring-a new kind of illness scoring system

Predicting the risk of inpatient hypoglycemia with machine learning using electronic health records

Clinical conditions during severe hypoglycemia in patients with diabetes

Predicting occurrences of acute hypoglycemia during insulin therapy in the intensive care unit

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD statement

Development and reporting of prediction models: Guidance for authors from editors of respiratory, sleep, and critical care journals

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and elaboration

Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis

Signatures of subacute potentially catastrophic illness in the ICU: Model development and validation

Ridge regression: Biased estimation for nonorthogonal problems

Pathophysiologic signatures of bloodstream infection in critically ill adults

Regression and time series model selection in small samples

Assessing model fit by crossvalidation

MIMIC-III, a freely accessible critical care database

MIMIC-III Waveform Database Matched Subset (version1.0)

Hypoglycaemia, hypothermia and shivering in man

Thermoregulatory responses to hyperinsulinemic hypoglycemia and euglycemia in humans

Hypoglycemia and body temperature

A physiological model of the effect of hypoglycemia on plasma potassium

Hypoglycemia, chronic kidney disease, and diabetes mellitus

Cardiorespiratory dynamics measured from continuous ECG monitoring improves detection of deterioration in acute care patients: A retrospective cohort study

Rare events in the ICU: An emerging challenge in classification and prediction

Hidden in plain sightreconsidering the use of race correction in clinical algorithms

Recalibrating the use of race in medical research

Understanding receiver operating characteristic (ROC) curves

Stress-induced hyperglycemia

Hypocalcemia: A pervasive metabolic abnormality in the critically ill

Acid-base disturbances in intensive care patients: Etiology, pathophysiology and treatment

Body temperature abnormalities in non-neurological critically ill patients: A review of the literature

Prevention of inpatient hypoglycemia with a real-time informatics alert

Prediction and prevention of treatment-related inpatient hypoglycemia

Predicting inpatient hypoglycaemia in hospitalized patients with diabetes: A retrospective analysis of 9584 admissions with diabetes

Development and validation of a prediction model for insulin-associated hypoglycemia in non-critically ill hospitalized adults

Development and validation of a machine learning model to predict near-term risk of iatrogenic hypoglycemia in hospitalized patients

Inpatient and outpatient technologies to assist in the management of insulin dosing

Subcutaneous insulin dosing calculators for inpatient glucose control

Evaluation of several electronic glycemic management systems

Safety and effectiveness of a computerized subcutaneous insulin program to treat inpatient hyperglycemia

NICE-SUGAR Study Investigators: Intensive versus conventional glucose control in critically ill patients

Multidisciplinary strategies to treat severe hypoglycemia in hospitalized patients with diabetes mellitus reduce inpatient mortality rate: Experience from an academic community hospital

We thank our colleagues in the University of Virginia Center for Advanced Medical Analytics for their helpful comments throughout the model development and validation process. We also thank Katy Krahn for her editorial input and Prof. Douglas E. Lake, challenger of sepsis, for many discussions and insights into predictive analytic monitoring.