key: cord-0904112-g3rf3f6f
authors: Wilfong, Erin M.; Lovly, Christine M.; Gillaspie, Erin A.; Huang, Li-Ching; Shyr, Yu; Casey, Jonathan D.; Rini, Brian I.; Semler, Matthew W.
title: Severity of illness scores at presentation predict ICU admission and mortality in COVID-19
date: 2021-01-25
journal: J Emerg Crit Care Med
DOI: 10.21037/jeccm-20-92
sha: 413b51d4df53e23d59558bcf1e0c303b6bf39ab0
doc_id: 904112
cord_uid: g3rf3f6f

BACKGROUND: The COVID-19 pandemic has overwhelmed hospital systems in multiple countries and necessitated caring for patients in atypical healthcare settings. The goal of this study was to ascertain if the conventional critical care severity scores qSOFA, SOFA, APACHE-II, and SAPS-II could predict which patients admitted to the hospital from an emergency department would eventually require intensive care. METHODS: This single-center, retrospective cohort study enrolled patients admitted to Vanderbilt University Hospital from the emergency room with symptomatic, confirmed COVID-19 infection between March 8, 2020 through May 15, 2020. Clinical phenotyping was performed by chart abstraction, and the correlation of the qSOFA, SOFA, APACHE-II, and SAPS-II scores for the primary endpoint of ICU admission and secondary endpoint of in-hospital mortality was evaluated. RESULTS: During the study period, 128 patients were admitted to Vanderbilt University Hospital from the emergency room with COVID-19. Of these, 39 patients eventually required intensive care; the remaining 89 were discharged from the medical ward. All severity of illness scores demonstrated at least moderate ability to identify patients who would die or require ICU admission. Of the three severity of illness scores assessed, the APACHE-II score performed best with an AUC of 0.851 (95% CI: 0.786 to 0.917) for identifying patient that would require ICU admission. No patient with an APACHE-II score at the time of presentation less than 8 or qSOFA of 0 required intensive care unit (ICU) admission. All patients with an APACHE-II score less than 10 or qSOFA score of 0 survived to hospital discharge. CONCLUSIONS: The APACHE-II score accurately predicts the eventual need for ICU admission. This may allow for risk-stratification of patients safe to treat in alternative health care settings and prognostic enrichment to accelerate clinical trials of COVID-19 therapies.

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has led to over 7.5 million global cases and 425,000 deaths due to coronavirus disease 2019 (COVID-19) as of June 13, 2020. An emerging challenge in the COVID-19 pandemic is identifying, among the patients presenting to the hospital, which patients are likely to require intensive care unit (ICU) and which can be managed without such intensive resources. Predicting ICU admission would help allocate clinical resources and allow prognostic enrichment in clinical trials. Various severity of illness scores has been developed to predict the risk of mortality at the time of ICU admission, including the sequential organ failure assessment (SOFA) (1), simplified acute physiology score (SAPS) II (2) , and acute physiology and chronic health evaluation (APACHE) II (3) . These scores have also demonstrated utility in evaluating disease severity outside of the ICU setting. SOFA score at the time of emergency department presentation predicts outcomes in severe sepsis (4) . The APACHE-II predicts mortality of acute pancreatitis at initial presentation (5) and the long-term mortality of patients admitted with a COPD exacerbation outside of the ICU (6) . Higher APACHE-II and SAPS-II scores also correlated with subsequent ICU admission in patients admitted to a tertiary intermediate care unit (7) . The APACHE-II score was recently shown to predict ICU mortality in COVID-19 (8), but its performance at the time of hospital presentation and its ability to predict ICU admission for patients with COVID-19 remain unknown. Further, the performance of these risk scores, relative to newly proposed COVID-specific markers of severity of illness such as the neutrophil:lymphocyte ratio, have yet to be evaluated (9) .

The performance of the SOFA, SAPS-II, APACHE-II severity of illness scores and the neutrophil:lymphocyte ratio were evaluated for their ability to predict ICU in a retrospective cohort of patients hospitalized for COVID-19 at a large academic medical center. We present the following article in accordance with the STROBE reporting checklist (available at http:// dx.doi.org/10.21037/jeccm-20-92).

This retrospective cohort study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and was approved by the Vanderbilt University institutional review board with wavier of informed consent (IRB#200537). Between March 8 th and May15 th , 2020, we enrolled all adult patients (18 years or older) who were admitted from the emergency department (ED) to Vanderbilt University Medical Center in Nashville, TN with symptomatic COVID-19, confirmed by SARS-CoV-2 testing. Patient outcomes were followed until June 12 th at 5pm. Patients readmitted within 72 hours of discharge were analyzed as a single admission.

Study personnel reviewed electronic health records to collect baseline patient characteristics including age, gender, race, ethnicity, body mass index, home medications, comorbidities, and smoking history. Active malignancy was defined as receipt of chemotherapy, radiotherapy, or surgery for malignancy within 45 days of admission. Vital signs and laboratory values were recorded. Missing data was analyzed by pairwise deletion. The primary outcome was ICU admission at any point during the inpatient hospitalization. Secondary outcomes were mortality and hospital length of stay. Patients who expired without ICU transfer based on limitations in care (e.g., a patient who expired after transfer from the hospital ward to inpatient hospice) were analyzed as not experiencing ICU transfer. Patients discharged to hospice were analyzed as having died. Patients who remained hospitalized at the end of the follow-up period were censored for analyses of mortality.

The SOFA (1), SAPS-II (2), APACHE-II (3), and qSOFA (10), were calculated using data collected within 24 hours of ED presentation. Calculation of SOFA scores substituted the oxygen saturation to fraction of inspired oxygen (S/F) ratio (11) in place of the PaO 2 to fraction of inspired oxygen (P/F) ratio. APACHE-II scores were calculated without inclusion of the PaO 2 term (12) and using bicarbonate instead of pH (13) . Scores were calculated blinded to patient outcome.

Sample size was estimated by the precision analysis. With the proposed sample size of 110, and an estimated ICU admission rate of 30%, the half-width of the two-sided 95% confidence interval (CI) for receiver operating characteristic (ROC) curve's area under the curve (AUC) was less than 12%. Continuous variables were reported as median and interquartile range (IQR). Categorical variables were reported as frequencies and proportions. Between-group comparisons were made with the Mann-Whitney rank-sum test for continuous variables and the Fishers exact test for categorical variables. Comparisons between AUCs were made using Delong's test. A two-sided P value <0.05 was used to indicate statistical significance without adjustment for multiple testing. ROC curves with 95% CI: for qSOFA score, SOFA score, SAPS-II score, APACHE-II score, and neutrophil:lymphocyte ratio were generated using R version 4.0 (14) and package pROC 1.16.2 (15) .

A patient flow diagram is shown in Figure 1 . One hundred twenty-eight patients with laboratory confirmed SARS-CoV-2 were admitted from the ED during the study period from March 8, 2020 to May 15, 2020. At the end of the follow-up period on June 12, 2020, 125 patients were discharged and three patients remained hospitalized, all of whom had already required ICU admission.

Overall, 39/128 (30.4%) of patients received ICU care. Table 1 shows baseline patient characteristics for patients who did and did not require ICU admission. The most common indication for ICU admission was hypoxemic respiratory failure (n=32, 82.1%). Other indications included altered mental status (n=2, 5.1%), arrhythmia (n=1, 2.6%) and increased nursing needs (n=4, 10.3%). Twenty-four patients (61.5%) were admitted directly to the ICU, while 15 (38.4%) patients were initially admitted to the ward and subsequently transferred to the ICU. The median time from presentation to ICU transfer was 3.0 days with IQR from 1.0 to 5.5 days.

In univariate analysis, patients who required ICU admission were more likely to have diabetes mellitus (38.5% vs. 19.1%, P=0.03), active malignancy (10.3% vs. 1.1%, P=0.03), or require supplemental oxygen at admission (79.5% vs. 40.4%, P<0.001). Gender and race were not significantly associated with risk of ICU admission. Figure 2A shows the ROC curves for the three severity of illness scores. The C-index for the APACHE-II score (AUC =0.851, 95% CI: 0.786 to 0.917) was higher than the SAPS-II (AUC =0.758, 95% CI: 0.671 to 0.844, P=0.009), SOFA score (AUC =0.730, 95% CI: 0.642 to 0.817, P=0.003), and qSOFA score (AUC 0.713, 95% CI: 0.630 to 0.797, P=0.004). The neutrophil:lymphocyte ratio alone had a C-index of 0.756, but the addition of the neutrophil:lymphocyte ratio to the APACHE-II did not significantly improve predictive ability of the APACHE-II score (P=0.445). No patient with an APACHE-II score <8 experienced ICU admission during hospitalization, and 18.2% of patients with an APACHE-II score between 8 and 10 experienced ICU admission. Similarly, no patient with qSOFA score 0 experienced ICU admission during hospitalization, and 26.4% of patients with qSOFA score 1 experienced ICU admission. All patients who did not require supplemental oxygen on admission but still experienced critical illness had APACHE-II scores greater than 10. Figure 2B ,C display the risk of ICU admission stratified by APACHE-II and qSOFA score, respectively. More patients had an APACHE-II score <8 than qSOFA score of 0 (35/128 vs. 19/128, P=0.21).

111 patients survived to hospital discharge, and 14 patients died. Three critically ill patients remained hospitalized at the end of follow-up. At the end of follow-up, the overall mortality was 10.9% for all patients and 30.5% for ICU patients.

All three models predicted in-hospital mortality. The SAPS-II score had the highest AUC (0.911, 95% CI: 0.856 to 0.966). The SAPS-II score did not perform better than either the APACHE-II (AUC =0.851, 95% CI: 0.766 to 0.936, P=0.072) or SOFA (AUC =0.826, 95% CI: 0.732 to 0.919, P=0.068), but did have a better AUC than the qSOFA (AUC =0.801, 95% CI: 0.692 to 0.911, P=0.028). The ROC curve of all three severity of illness scores is shown in Figure 3A . Figure 3B ,C display the mortality risk stratified by APACHE-II and qSOFA score, respectively. No patient with an APACHE-II score less than 10 or qSOFA score of 0 died.

This single-center cohort study found that, among all patients with COVID-19 admitted from the ED, the APACHE-II score more accurately predicted subsequent ICU admission than the SOFA, SAPS-II, or qSOFA score. Notably, both the APACHE-II and SOFA performed well using previously described modifications that obviated the need for frequently unavailable arterial blood gas measurements. We substituted S/F for the SOFA P/F value (11) , eliminated the PaO 2 term from the APACHE-II (12), and used a serum bicarbonate substitution for the APACHE-II pH term (13) . In this cohort, patients with either a low APACHE-II score (<8) or qSOFA score (0) did not require ICU admission. All scores accurately predicted mortality. As previously reported, advanced age and history of diabetes mellitus were associated with poor outcomes. While this cohort did not identify non-Caucasian race as an independent risk factor for ICU admission or death among this cohort of patients hospitalized with COVID-19, the proportion of non-Caucasian requiring hospitalization was twice as high as observed in historical, institutional cohorts (16, 17) . The finding that APACHE-II scores can accurately predict subsequent ICU admission has two basic applications.

First, when making decisions regarding allocation of hospital resources, the APACHE-II score or qSOFA may identify patients with COVID-19 at low risk for ICU admission and death, who might be safely treated in lower acuity environments. Conversely, patients with COVID-19 and high APACHE-II scores presenting to facilities with limited ICU services might benefit from early transfer to a tertiary facility. APACHE-II score may have performed better than qSOFA, SOFA, and SAPS-II because APACHE-II incorporates a broader assessment of chronic comorbidities, which may influence both the severity of COVID-19 and an individual's physiologic reserve. It is notable that, while hypoxemia at the time of presentation is highly associated with the need for ICU admission, six patients on room air during the first 24 hours of admission required ICU care during their hospitalizations. The APACHE-II score predicted an increased risk of ICU admission in all six cases.

Second, a significant challenge for designing randomized therapeutic trials in COVID-19 is that the majority of patients will improve without treatment, which threatens to dilute outcome events, increases the required sample size, and increases the number of patients exposed to toxicities without potential benefit. Clinical trial outcomes such as death and mechanical ventilation occur predominantly in ICU patients. Thus, enrolling COVID-19 patients at risk for ICU admission into early treatment trials may prognostically enrich trials by increasing event rates, allowing smaller sample sizes, and improving the benefit/risk ratio for participants.

This study has several considerations. All studies are prone to bias, but this was mitigated through the use of objective inclusion criteria (selection bias), data collection while blinded to patient outcome (outcome bias), pre-specified modeling approaches (over-fitting bias), and sample size determination to avoid type II error. Conduct at a single tertiary care center, small sample size, and lack of a validation cohort may, however, limit generalizability. Additionally, nearly 50% of patients did not have a C-reactive protein, ferritin, or d-dimer measured at admission, which precluded additional biomarker analysis. If validated in larger cohorts, the ability of APACHE-II scores to predict ICU admission and mortality could have significant implications for patient care and clinical trials during the ongoing COVID-19 pandemic.

The APACHE-II score calculated within 24 hours of admission accurately predicts the eventual need for ICU care, and no patients with APACHE-II score <8 or qSOFA score of zero required ICU care. This finding could allow for safe triaging of patients to alternative care sites at times of high healthcare resource utilization and prognostically enrich future therapeutic clinical trials. 

The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine

A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study

APACHE II: a severity of disease classification system

The Sequential Organ Failure Assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation

Comparison of BISAP, Ranson's, APACHE-II, and CTSI scores in predicting organ failure, complications, and mortality in acute pancreatitis

APACHE II predicts long-term survival in COPD patients admitted to a general medical ward

Outcomes of Emergency Medical Patients Admitted to an Intermediate Care Unit With Detailed Admission Guidelines

Acute Physiology and Chronic Health Evaluation II Score as a Predictor of Hospital Mortality in Patients of Coronavirus Disease

Neutrophil-to-lymphocyte ratio and lymphocyte-to-C-reactive protein ratio in patients with severe coronavirus disease 2019 (COVID-19): A meta-analysis

Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)

Derivation and validation of Spo2/Fio2 ratio to impute for Pao2/Fio2 ratio in the respiratory component of the Sequential Organ Failure Assessment score

Clinical Characteristics and Outcomes Are Similar in ARDS Diagnosed by Oxygen Saturation/Fio2 Ratio Compared With Pao2/Fio2 Ratio

New scoring system (APACHE-HF) for predicting adverse outcomes in patients with acute heart failure: evaluation of the APACHE II and Modified APACHE II scoring systems

R: A language and environment for statistical computing. Foundation for Statistical Computing

Balanced Crystalloids versus Saline in Noncritically Ill Adults

Balanced Crystalloids versus Saline in Critically Ill Adults

Neutrophil:lymphocyte ratio (NLR)

Oxygen saturation to percent inspired oxygen ratio (S/F) 351.9

†, body mass index was missing for 6 patients (4.2%); 5 in the non-ICU group and 1 in the ICU group. ‡, estimated duration of symptoms missing for 11 patients, 8 in the non-ICU and 3 ICU patients

This work was supported by CTSA award No. UL1TR000445 (EMW, BIR) from the National Center for Advancing Translational Sciences. EMW was supported by NIH T32HL087738. CML was supported in part by NCI P30-CA086485. JDC was supported in part by the NIH (2T32HL087738-12, K12HL133117) . MWS was supported in part by the National Heart Lung and Blood Institute (K23HL143053). Its contents are solely the