key: cord-0796623-twv2g97d
authors: Artero, Arturo; Madrazo, Manuel; Fernández-Garcés, Mar; Muiño Miguez, Antonio; González García, Andrés; Crestelo Vieitez, Anxela; García Guijarro, Elena; Fonseca Aizpuru, Eva María; García Gómez, Miriam; Areses Manrique, María; Martinez Cilleros, Carmen; Fidalgo Moreno, María del Pilar; Loureiro Amigo, José; Gil Sánchez, Ricardo; Rabadán Pejenaute, Elisa; Abella Vázquez, Lucy; Cañizares Navarro, Ruth; Solís Marquínez, Marta Nataya; Carrasco Sánchez, Francisco Javier; González Moraleja, Julio; Montero Rivas, Lorena; Escobar Sevilla, Joaquín; Martín Escalante, María Dolores; Gómez-Huelgas, Ricardo; Ramos-Rincón, José Manuel
title: Severity Scores in COVID-19 Pneumonia: a Multicenter, Retrospective, Cohort Study
date: 2021-02-11
journal: J Gen Intern Med
DOI: 10.1007/s11606-021-06626-7
sha: 55c2973e4a9dff6615f6b3883a64f374cd5c8f9a
doc_id: 796623
cord_uid: twv2g97d

BACKGROUND: Identification of patients on admission to hospital with coronavirus infectious disease 2019 (COVID-19) pneumonia who can develop poor outcomes has not yet been comprehensively assessed. OBJECTIVE: To compare severity scores used for community-acquired pneumonia to identify high-risk patients with COVID-19 pneumonia. DESIGN: PSI, CURB-65, qSOFA, and MuLBSTA, a new score for viral pneumonia, were calculated on admission to hospital to identify high-risk patients for in-hospital mortality, admission to an intensive care unit (ICU), or use of mechanical ventilation. Area under receiver operating characteristics curve (AUROC), sensitivity, and specificity for each score were determined and AUROC was compared among them. PARTICIPANTS: Patients with COVID-19 pneumonia included in the SEMI-COVID-19 Network. KEY RESULTS: We examined 10,238 patients with COVID-19. Mean age of patients was 66.6 years and 57.9% were males. The most common comorbidities were as follows: hypertension (49.2%), diabetes (18.8%), and chronic obstructive pulmonary disease (12.8%). Acute respiratory distress syndrome (34.7%) and acute kidney injury (13.9%) were the most common complications. In-hospital mortality was 20.9%. PSI and CURB-65 showed the highest AUROC (0.835 and 0.825, respectively). qSOFA and MuLBSTA had a lower AUROC (0.728 and 0.715, respectively). qSOFA was the most specific score (specificity 95.7%) albeit its sensitivity was only 26.2%. PSI had the highest sensitivity (84.1%) and a specificity of 72.2%. CONCLUSIONS: PSI and CURB-65, specific severity scores for pneumonia, were better than qSOFA and MuLBSTA at predicting mortality in patients with COVID-19 pneumonia. Additionally, qSOFA, the simplest score to perform, was the most specific albeit the least sensitive.

At the end of 2019, a novel coronavirus was identified as the cause of an outbreak of pneumonia cases in Wuhan, Hubei Province, China. This virus, now known as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), rapidly spread around the world, causing a pandemic that has affected more than 61·8 million people as of 30 November 2020. [1] [2] [3] The disease that it produces is called coronavirus disease 2019 and is characterized by a wide spectrum of infectious symptoms that range from mild to severe, with acute respiratory distress syndrome (ARDS) as its primary complication. [4] [5] [6] Hospitalization is required in 6 to 20% of patients. 7, 8 Mortality among hospitalized patients is high, ranging from 11 to 28%. 4, 5 Therefore, early and simple identification upon admission to hospital of patients who may have poor outcomes would be of considerable value and may help lead to prompt treatment and an optimized use of resources.

Several prognostic scores have been developed to identify elevated risk of death in patients with community-acquired pneumonia (CAP). Two of them, the Pneumonia Severity Index (PSI) and the CURB-65 score, are well-validated scores for supporting pneumonia prognoses. 9, 10 The MuLBSTA score is based on six parameters (Multilobar infiltration, hypo-Lymphocytosis, Bacterial coinfection, Smoking history, hypertension, and age) routinely measured in hospitals. MuLBSTA is a new prognostic tool for patients hospitalized with viral pneumonia 11 and has been suggested as a severity score for patients with COVID-19. 5 The qSOFA, based on the sepsis-3 criteria, 12 has proven to be a useful tool in the emergency department, 13, 14 non-intensive-care-unit (ICU) wards, [15] [16] [17] and even at home 18, 19 for establishing a prognosis in patients with documented or suspected infection with different foci. Recent works indicate that qSOFA is also a useful tool for identifying patients with poor prognoses in viral infections such as influenza. [20] [21] [22] In addition, qSOFA has shown good performance in predicting prognosis in pneumonia, both in the emergency department [23] [24] [25] [26] [27] and in wards. 28 The advantage of the qSOFA score is that the variables are clinical and thus laboratory tests are not required. 12 

In this study, we aim to ascertain the applicability and prognostic prediction value of the PSI, CURB-65, MuLBSTA, and qSOFA severity scores in COVID-19 patients with pneumonia admitted to hospital.

This work is a multicenter retrospective cohort study of patients with SARS-CoV2 infection, aged ≥18 years, and with CAP who were hospitalized between March 1 and May 28, 2020, in Spain and included in the aforementioned SEMI-COVID-19 Registry. 29 The registry is an initiative of the Spanish Society of Internal Medicine (SEMI, for its initials in Spanish), and its characteristics have been described elsewhere. 29 Data are collected retrospectively and include epidemiological and clinical characteristics, such as comorbidities, symptoms, physical examination findings, laboratory and diagnostic imaging test results, and clinical outcomes.

COVID-19 pneumonia was defined as compatible symptoms (cough, dyspnea, need for respiratory support, fever, or crackling or rales on auscultation), pulmonary infiltrates, or consolidation detected by a chest X-ray or computerized tomography (CT) scan, and a positive result on either a polymerase chain reaction test of a nasopharyngeal sample or a serology test for COVID-19 antibodies.

We defined PSI, CURB-65, MuLBSTA, and qSOFA as originally described [9] [10] [11] [12] . PSI is a scale with 19 variables with different point values. 9 A cut-off of 91 points (risk class IV-V) was considered positive for a poor prognosis 24-28 . We considered CURB-65 to be predictive of poor prognosis when three of the five variables (confusion, BUN >7 mmol/L, respiratory rate ≥30 bpm, systolic blood pressure <90 mmHg or diastolic blood pressure ≤60 mmHg, and age ≥65 years) were positive [21] [22] [23] [24] . The MuLBSTA score is a newly designed score for viral pneumonia 11 that considers the following variables: multilobar infiltrates (5 points), lymphocytes ≤0·8 x10e9/ml (4 points), bacterial infection (4 points), active smoker (3 points), prior smoker (2 points), hypertension (2 points), and age ≥60 years (2 points), with a cut-off point of 12 points or more is related with poor prognosis, as defined in the original study. 11 qSOFA was considered positive when two of the three variables (altered mental status, respiratory rate ≥22 brpm, and/or systolic blood pressure <100 mmHg) were met, according to the Sepsis-3 definition. 12 

The primary outcome measure was all-cause in-hospital mortality. As secondary outcomes, we analyzed admission to intensive care unit (ICU) and use of mechanical ventilation.

After a preliminary descriptive analysis of the data (SD, means, and percentages), we compared the differences between categorical variables using the chi-square test and continuous variables using Student's t test or analysis of variance.

Sensitivity and specificity of the scores were calculated using the cut-off points of poor prognosis proposed in the validation study of the score (MuLBSTA and qSOFA) or habitual use in other studies (PSI and CURB-65), as is detailed in the definition of variables. In the AUROC, all scores were analyzed by points, as a graded risk assessment system. When data for the calculation of the score were missing, the case was excluded. The SPSS statistical package version 22.0 (SPSS, Chicago, IL) was used for the statistical study and the Epidat version 3.1 program was used for comparing the AUROCs of the different scales. P values <0.05 were considered statistically significant. 

A total of 10,238 patients were included in this study. The patient inclusion flowchart can be seen in Figure 1 . Epidemiological and clinical data are found in Table 1 . The mean age was 66.6 ± 16.2 years and 57.9% of patients were male. A total of 5830 (56.9%) patients had an age-adjusted Charlson Comorbidity Index of 3 or more points. The most frequent comorbidities were hypertension (49.2%), diabetes (18.8%), and COPD (12.2%). ARDS and acute kidney injury were the most common complications (34.7% and 13.9%, respectively).

In-hospital mortality was 20.9% (2135 cases). A total of 907 (8.9%) patients were admitted to ICU. The mean length of hospital stay was 11.2 ± 9.2 days. As expected, the patients who died were older and had more comorbidities and complications. Prognostic scores were more frequently positive in deceased patients, as can be seen in Table 1 . Patients who died also had a longer mean hospital stay (9.7 vs 11.6 days, p <0.001) and were more frequently admitted to the ICU (17.6% vs 6.6%, p <0.001).

PSI was calculated in 9261 (90.5%) cases, CURB-65 and qSOFA were calculated in 9887 (96.6%) cases, and MuLB-STA was calculated in 9505 (92.8%) cases. The PSI and CURB-65 scores showed no statistical differences upon comparison (AUROC 0.835 vs 0.825, p=0.112) for mortality and were higher than the qSOFA and MuLBSTA scores (AUROC 0.728 and 0.715, respectively, p<0.001 for both compared to PSI and CURB-65). These findings can be seen in Table 2 and Figure 2 . The qSOFA score showed no differences compared to the MuLBSTA score (AUROC 0.728 vs 0.715, p=0.102) and was the most specific score (95.72%), as can be seen in Table 3 , but it had a lower sensitivity. PSI risk class IV-V had the highest sensitivity (84.12%) but a lower specificity than the qSOFA and MuLBSTA scores (72.25% vs 95.72% and 91.23%, p <0.001 for both scores).

The results of the secondary outcomes admission to ICU or use of mechanical ventilation are shown in Table 2 . As we can see, in both outcomes, MuLBSTA and qSOFA were superior to PSI and CURB-65, which had no difference between them. However, the prognostic accuracy of the scores for these outcomes was suboptimal.

To our knowledge, this is the largest study evaluating prognostic scores in patients with COVID-19 pneumonia. The inhospital mortality observed in this multicenter study in Spain was high (20.9%). This could be due to the fact that the mean age of patients was 66.6 years and there was a high frequency of comorbidities. The information obtained at admission allowed us to compare severity scores for identifying patients at high risk of in-hospital death. PSI and CURB-65 had better prognostic accuracy, with an AUROC of 0.835 and 0.825, respectively.

Mortality in our results was higher than that in other cohorts as Liang et al. 33 . In our cohort, the mean age was 66.6 years and 56.9% patients had an ageadjusted Charlson Comorbidity Index of 3 or more points, in consonance with cohorts with similar mortality (mean age 65.9, median Charlson Comorbidity Index of 3 (1-5)) 33 ; meanwhile, the cohorts with lower mortality had lower mean age (48.9 and 56.9 years 30,31 ) and less comorbidities. 30, 31 PSI was initially designed to help practitioners identify which low-risk patients with CAP could be safely treated in an outpatient setting, 9 although it has subsequently been used to assess post-discharge mortality for those treated as inpatients. The capability of PSI to predict hospital mortality in our study was similar than another study in COVID-19 pneumonia (AUROC 0.85). 32 This AUROC is similar or even slightly better than that of previous studies in CAP, in which PSI presented an AUROC of 0.778 to predict 30-day mortality or transfer to ICU 28 and 0.812 to predict 30-day mortality. 34 Our findings confirm that PSI is quite a good score for assessing the risk of in-hospital mortality for patients admitted with COVID-19 pneumonia.

CURB-65 was developed to stratify hospitalized patients with CAP into mortality risk groups, with a primary outcome measure of 30-day mortality. 10 It has shown a diagnostic performance for mortality similar to the PSI in CAP. 24 The diagnostic performance of CURB-65 for mortality found in our study was comparable to that found by Su et al. 35 (AUROC 0.85) in their study on 116 patients with COVID-19 in China, whose primary outcome measure was intensive respiratory or vasopressor support. These results are consistent with other COVID-19 studies with mortality as primary outcome (AUROC 0.85) 32 , and better than those with composed . MuLBSTA is a recently described severity score for 90-day mortality in patients with viral pneumonia. 11 It has been shown to perform better than CURB-65 in viral pneumonia, mainly caused by influenza virus, human rhinovirus, and respiratory syncytial virus. 11 Chen et al. suggested using MuLBSTA as a severity score for COVID-19 in their description of 99 patients in Wuhan, China 5 , because the patients who died had a high percentage of variables included in this score. In our results, the AUROC of MuLBSTA for mortality (0.715) showed lower sensitivity compared to the other scores as well as when compared to the original work (0.715 vs 0.773, p <0.001). Further studies are needed in order to prove the suitability of the MuLBSTA score.

In our study, qSOFA showed a lower sensitivity than the PSI and CURB-65 scores (AUROC 0.728 vs 0.835 and 0.825, p <0.001 for both scores). These results were in line with the results found by Su et al. 35 in their study previously commented. The superiority of CRB-65, a score comparable to CURB-65, versus qSOFA for prediction of intensive respiratory or vasopressor support (AUROC 0.81 vs 0.70, p 0.02) was observed in this study. It was also similar to other studies previously commented, with primary outcome of mortality (AUROC 0.73) 32 . In this study, PSI and CURB-65 were also superior to qSOFA (AUROC 0.85 both scores vs. 0.73) 32 . Jang et al. 31 , in a study with 110 patients, showed an AUROC of 0.779 for the primary outcome of mortality at day 28, and 0.760 for "critical outcomes," composed by transfer to ICU, septic shock, or ARDS. Our results were superior to that showed by Lalueza et al. in a retrospective study with 237 patients 33 , with an AUROC of 0.69 for respiratory failure or death. Other studies have also shown that the accuracy of qSOFA in patients with COVID-19 was limited, 37,38 although these studies included a small number of patients. However, qSOFA has been related to mortality in COVID-19 in another study with 191 patients (OR 12 (CI 95% 5.06-28.43)). 4 In our work, the PSI and CURB-65 scores showed sensitivity and specificity values similar to those found in other studies on CAP. 23, 28, 34 However, qSOFA sensitivity was lower than the other scores and lower for CAP (53 to 70.1%). 23, 26 qSOFA had a higher specificity than the other scores in our study (95.72% vs 72.25%, 90.68%, and 91.23% for PSI, CURB-65, and MuLBSTA, respectively). These results for qSOFA are consistent with the findings of Su et al., 35 who observed a similar specificity for qSOFA and CURB-65 (98.9% and 96.7%, respectively). Despite its lower sensitivity, qSOFA has the benefit of not requiring laboratory test results.

In the study by Su et al., 35 the CRB score was also analyzed, which also has the benefit of not requiring laboratory results. In that study, no difference was found between CRB and qSOFA. 35 These two scores had a lower accuracy than CRB-65 and CURB-65, suggesting that age is an important risk factor for mortality. Indeed, age has been shown to be an independent risk factor for mortality in patients with COVID-19. 39, 40 This study has some limitations. First, due to its retrospective design, some possible confounding variables were not recorded and thus we could not calculate prognostic scores in 3.4 to 9.5% of cases. Second, we focused only on hospitalized patients and as such, we cannot be certain that our findings can be extrapolated to outpatients. Third, by excluding patients still hospitalized as of May 29, 2020, the case fatality rate in our study does not reflect the true mortality rate of COVID-19. Fourth, this study was conducted in multiple centers in Spain and its results may not be applicable to other settings with different populations or healthcare systems. Fifth, the results of the secondary outcomes may be biased by the availability of ICU beds and ventilators in the first months of the pandemic. It is important to note that although these scores could help physicians identify patients with COVID-19 pneumonia at admission to hospital who have different risk levels for death, there are other risk factors specific to COVID-19 that should be considered, such as lymphopenia or D-dimer, 4 which are not analyzed in these scores.

Zhou et al 4 showed that a D-dimer of >1 μg/ml was associated with mortality (OR 18.42 (95% CI 2.64-128.55)). In a retrospective study with 4389 patients 41 , anticoagulation was associated with lower mortality and intubation among hospitalized COVID-19 patients. However, our study is focused on prognostic scores, and regrettably we did not analyze the effect of treatments in mortality.

Recently, more than 22 specific scores for COVID-19 have been developed 42 . The CALL scoring system, composed by underlying comorbidity, advanced age, higher LDH, and lower lymphocyte count, showed an AUROC of 0.91 for progression of COVID-19 43 , and is simpler than the 12parameter MuLBSTA score proposed by Guo et al. 11 Liang et al. 30 developed and validate other more complex system, the GRAM-COVID, a clinical risk score to predict the development of critical illness among hospitalized patients infected with COVID-19. 30 Performance of this risk score was satisfactory with AUC-based precision in the development and validation cohorts of 0.88. But both scores 30, 43 are limited by a modest sample size for constructing the risk score and a relatively small sample for validation. The PRIORITY score 42 has been developed by the SEMI-COVID-19 Network to predict critical illness, in a retrospective study with 10433 patients. This score that includes epidemiological and clinical variables has shown an AUROC of 0.795 in the validation cohort.

However, development of new scores always needs an external validation and other studies to prove its usefulness. Therefore, we believe that using some well-established scores such as CURB-65 and PSI, used in pneumonia, or a general and simple sepsis score as qSOFA, will be in benefit of the clinician and the patients.

PSI and CURB-65, two severity scores specific to pneumonia, performed better than qSOFA and MuLBSTA for predicting all-cause in-hospital death for patients with COVID-19 pneumonia. qSOFA, the simplest score to calculate, was the most specific, albeit the least sensitive. The four scores had a suboptimal performance for the secondary outcomes admission to ICU or use of mechanical ventilation.

Outbreak of pneumonia of unknown etiology in Wuhan, China: The mystery and the miracle

Hypothesis for potential pathogenesis of SARS-CoV-2 infection-a review of immune changes in patients with viral pneumonia

World Health Organization

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study

Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study

Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan, China

COVID-19 Testing, Epidemic Features, Hospital Outcomes, and Household Prevalence

Characteristics of and important lessons from the Coronavirus Disease 2019 (COVID-19) outbreak in China. Summary of a Report of 72314 cases from the Chinese Center for Disease Control and Prevention

The hospital admission decision for patients with community acquired pneumonia

Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study

Clinical Features Predicting Mortality Risk in Patients With Viral Pneumonia: The MuLBSTA Score

Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)

Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department

Sepsis Clinical Criteria in Emergency Department Patients Admitted to an Intensive Care Unit: An External Validation Study of Quick Sequential Organ Failure Assessment

The quick Sepsisrelated Organ Failure Assessment (qSOFA) is a good predictor of inhospital mortality in very elderly patients with bloodstream infections. A retrospective observational study

Frequency and significance of qSOFA criteria during adult rapid response team reviews: A prospective cohort study

A comparison of pre ICU admission SIRS, EWS and q SOFA scores for predicting mortality and length of stay in ICU

Prognostic value of prehospital quick sequential organ failure assessment score among patients with suspected infection

Use of prehospital qSOFA in predicting in-hospital mortality in patients with suspected infection: A retrospective cohort study

Quick-SOFA score>/2 predicts prolonged hospital stay in geriatric patients with influenza infection

Predictors of mortality of influenza virus infections in a Swiss Hospital during four influenza seasons: Role of quick sequential organ failure assessment

Quick-SOFA score to predict mortality among geriatric patients with influenza in the emergency department

External Validation of the qSOFA Score in Emergency Department Patients With Pneumonia

New Sepsis Definition (Sepsis-3) and Community-Acquired Pneumonia Mortality: a validation and clinical decision-making stuy

Efficacy of quick Sequential Organ Failure Assessment with lactate concentration for predicting mortality in patients with community-acquired pneumonia in the emergency department

Efficacy of the quick sequential organ failure assessment for predicting clinical outcomes among community-acquired pneumonia patients presenting in the emergency department

Stratified and prognostic value of admission lactate and severity scores in patients with community-acquired pneumonia in emergency department

Efficacy and accuracy of qSOFA and SOFA scores as prognostic tools for community-acquired and healthcare-associated pneumonia

Clinical characteristics of patients hospitalized with COVID-19 in Spain: results from the SEMI-COVID-19 Registry

Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19

Prognostic accuracy of the SIRS, qSOFA, and NEWS for early detection of clinical deterioration in SARS-CoV-2 infected patients

Comparison of severity scores for COVID-19 patients with pneumonia: a retrospective study

Utilidad de las escalas de sepsis para predecir el fallo respiratorio y la muerte en pacientes con COVID-19 fuera de las Unidades de Cuidados Intensivos

Sequential organ failure assessment score is an excellent operationalization of disease severity of adult patients with hospitalized community acquired pneumonia -results from the prospective observational PROGRESS study

Comparison of CRB-65 and quick sepsisrelated organ failure assessment for predicting the need for intensive respiratory or vasopressor support in patients with COVID-19

Development and Validation of the Quick COVID-19 Severity Index: A Prognostic Tool for Early Clinical Decompensation

Critically ill SARS-CoV-2-infected patients are not stratified as sepsis by the qSOFA

National Early Warning Score 2 (NEWS2) on admission predicts severe disease and in-hospital mortality form COVID-19. A prospective cohort study

Clinical features of COVID-19 in elderly patients: A comparison with young and middle-aged patients

Clinical characteristics of coronavirus disease 2019 (COVID-19) in China: A systematic review and metaanalysis

Anticoagulation, bleeding, mortality, and pathology in hospitalized patients with COVID-19

Predicting critical illness on initial diagnosis of COVID-19: development and validation of the PRIORITY model for outpatient applicability

Prediction for Progression Risk in Patients With COVID-19 Pneumonia: The CALL Score

We want to thank the contribution of the SEMI-COVID-19 Network for data retrieval.