key: cord-0917490-rnmy4xko
authors: Doğanay, Fatih; Ak, Rohat
title: Performance of the CURB‐65, ISARIC‐4C and COVID‐GRAM scores in terms of severity for COVID‐19 patients
date: 2021-09-03
journal: Int J Clin Pract
DOI: 10.1111/ijcp.14759
sha: 4dcf4f8a5797823b9fd2a7ad57e56e19e4dc3e6f
doc_id: 917490
cord_uid: rnmy4xko

BACKGROUND: In the COVID‐19 pandemic, difficulties have been experienced in the provision of healthcare services because of excessive patient admissions to hospitals and emergency departments. It has become important to use clear and objective criteria for the early diagnosis of patients with high‐risk classification and clinical worsening risk. OBJECTIVE: The aim of this study was to assess the prognostic accuracy of CURB‐65, ISARIC‐4C and COVID‐GRAM scores in patients hospitalised for COVID‐19 and to compare the scoring systems in terms of predicting in‐hospital mortality and intensive care unit requirement. METHODS: The files of all COVID‐19 patients over the age of 18 who were admitted to the emergency department and hospitalised between September 1, 2020 and December 1, 2020 were retrospectively scanned. The area under the receiver operating characteristic curve and Youden J Index were used to compare scoring systems for predicting in‐hospital mortality and intensive care requirement. RESULTS: There were 481 patients included in this study. The median age of the patients was 67 (52‐79). In terms of in‐hospital mortality, the AUC of CURB‐65, ISARIC‐4C and COVID‐GRAM were 0.846, 0.784 and 0.701 respectively. In terms of intensive care requirement, the AUC of CURB‐65, ISARIC‐4C and COVID‐GRAM were 0.898, 0.797 and 0.684 respectively. In our study, Youden's J indexes of CURB‐65, ISARIC‐4C and COVID‐GRAM scores were found to be 0.59, 0.27 and 0.01 respectively, for mortality prediction of COVID‐19 patients. Whereas Youden's J indexes were found to be 0.63, 0.26 and 0.01 respectively for determining intensive care requirement. CONCLUSIONS: Among the scoring systems assessed, CURB‐65 score had better performance in predicting in‐hospital mortality and ICU requirement in COVID‐19 patients. ISARIC‐4C has been found successful in identifying low‐risk patients and the use of the ISARIC‐4C score with CURB‐65 increases the accuracy of risk assessment.

patients is related to acute respiratory failure, especially in those with comorbidities. [4] [5] [6] It is important to use clear and objective criteria for risk stratification and for early diagnosis of patients who are at a high risk of clinical worsening. Scoring systems used for this purpose are valuable tools that help clinicians predict outcomes and guide treatmentrelated decisions. 7 Among these scoring systems, the CURB-65 score has been developed as a clinical prediction rule suitable for use in the ED. In addition, this score is used to predict the prognosis of patients using variables that can be easily measured during the initial evaluation. 8 It has been reported that this score, which is widely used for pneumonia patients, also has a strong predictive value in COVID-19 patients. 9 Scoring systems that are specific to COVID-19 patients have also been developed. Among these, the

Clinical Characterization Consortium (ISARIC-4C) score is derived from a prospective observational cohort study based on COVID-19 patients admitted to 260 hospitals in England, Scotland and Wales and provides information about the prognosis of patients. 10 According to this scoring system, it has been reported that patients in the low risk group (mortality rate 1%) can be followed on an outpatient basis, while patients in the medium risk group (mortality rate 10%) should be admitted to the hospital. 10 Similarly, the purpose of the COVID-GRAM score is to help prediction of the COVID-19 patients' risk rate for critical illness. 11 It has been suggested that monitorised follow-up of patients in the lowrisk group for critical illness will be sufficient, and that patients in the high-risk group should receive more aggressive treatment. 11 The aim of this study was to determine the prognostic accuracy of CURB-65, ISARIC-4C and COVID-GRAM scores in patients hospitalised for COVID-19 and to compare the scores with each other in this regard.

This retrospective observational study was carried out in the ED 

All COVID-19 patients over the age of 18 who were hospitalised be- were not included in this study (Table S2) . Additionally, patients who needed CPR in the ED, who died in the ED and who were pregnant

were not included in this study.

For the patients who were included in this study, their age; gender; CT findings; symptoms; Glasgow coma scale (GCS) score; chronic 

The patient's hospitalisation outcome, ie survivor and non-survivor groups, represent in-hospital mortality. The primary outcome was to determine the diagnostic accuracy of each scoring system for in-hospital mortality. The patient's ED outcome, that is, IU and ICU groups, represents the first unit where the patient is hospitalised from the ED. The secondary outcome was to determine the diagnostic accuracy of each scoring system for ICU requirement.

Outcomes were retrospectively assessed by reviewing the hospital medical database. Figure 3 .

CURB-65 and ISARIC-4C scores of the patients were determined according to the points they received from each category (Table S1) .

First, the COVID-GRAM risk score was determined using the following formula, "α = (CT abnormality × 1.2205) + (Age × 0.0276) + Comparisons of mortality and ICU groups were analysed using

Mann-Whitney U test for numerical data and chi-square test for categorical data. Numerical data were reported as medians and interquartile ranges (25th-75th), while categorical data were reported as frequencies and percentages (Tables 1 and 2 ).

Among the scoring systems, the COVID-GRAM score is divided into three risk groups, ISARIC-4C into four risk groups and the CURB-65 into five risk groups. When comparing the scores, all three were standardised into three risk groups as low, medium and high.

Based on the literature, this was done to avoid being unstandardised because of the different number of groupings. 10, 11, 15, 16 medium (1.7%-40.4%), high (40.4%-100%); for CURB-65 low (0 and 1), medium (2), high (3) (4) (5) ; and for ISARIC-4C low (0-3), medium (4-8),

high (9) (10) (11) (12) (13) (14) (15) (16) (17) (18) (19) (20) (21) .

To examine the accuracy of the diagnostic accuracy of the scoring systems in detail, a receiver operating characteristic (ROC) analysis was performed and area under the curve (AUC) was calculated.

were calculated at a criterion >1, as has been suggested with guidelines. 15, 16 YJI was calculated as well as AUC to assess the predictive accuracy of the scoring systems. Comparisons between the AUC values of the scoring systems were analyzed by the DeLong method. 17 A P < .05 was considered statistically significant.

This study was conducted with data from 481 patients after applying the inclusion and exclusion criteria. In the groups compared in the study, there were 361 patients in the surviving group, 120 patients in the non-surviving group, 396 patients in the IU group, and 85 patients in the ICU group. When the in-hospital mortality outcome was evaluated by gender, 54 women (45%) and 66 men (55%) died (Table 1) .

The median age of the population included in the study was 67 (52-79). The median age of the survivor group was 63 (49-76) and

the non-survivor group was 75 (66-82) ( Table 2) . Among the chronic diseases, there was a significant difference between the mortality groups for CHF (P = .001), CRF (<0.001), CND (P = .014), Alzheimerdementia (P < .001) ( Table 1 ). There was a significant difference between the ICU requirement groups for CHF (P = .001) and CAD (P = .038) ( were significant differences (P < .001) for both grouping methods ( 100%, 0.01 for COVID-GRAM respectively ( Figure 2 , Table 3 ).

Our study showed that the CURB-65 score assessed on admission to the ED in COVID-19 patients is more accurate than the COVID-GRAM and ISARIC-4C scores in predicting ICU admission and in-hospital mortality risk. Although the sensitivity of the COVID-GRAM score was higher than the CURB-65 and ISARIC-4C scores, the diagnostic accuracy of the CURB-65 score was better than the other two scores.

Quick and accurate identification of critically ill patients ensures the appropriate and correct use of medical resources. Implementing scoring systems could facilitate more effective evaluation by ED physicians and ICU physicians in identifying critically ill patients.

In the early stages of the pandemic, there was no specific score for COVID-19, for this reason, known scores such as CURB-65 were utilised. 18 The CURB-65 score has been used as a safe predictor of 30-day mortality in patients with pneumonia for many years. 8 It also helps clinicians make the decision to admit or discharge such pa- and whose primary outcome criteria were respiratory or vasopressor support. 21 These results are also consistent with other COVID-19 studies where mortality is the primary outcome. 22 CURB-65 score has been successful in predicting low-risk as well as high-risk, as can be seen in Figure 3A ,D and in Table 3 . For the low-risk patients, the numbers of false-negative cases are more than ISARIC-4C, but the numbers of true positive cases are ~2.7 times more than the ISARIC-4C (Table 3 ). The majority of patients defined as the medium risk (65.9%-81.7%) with CURB-65 are in the surviving and IU groups. The CURB-65 score giving the lowest numbers of falsenegative cases in the high-risk patients is the reason that it is the most successful score in classifying the high-risk patients correctly ( Figure 3 , Table 3 ).

Recently, more than 22 specific scoring systems have been developed for COVID-19. 24 Index (qCSI) scores were compared with each other in terms of in-hospital mortality, and superiority of the scores to each other was not determined. 25 In another study, NLR and COVID-GRAM were compared with each other in mortality prediction for COVID-19 patients, and no difference was found between these two predictors (AUC of 0.65 and 0.66 respectively). 26 As can be seen in Tables 3 and in Figure 3C ,F, mortality and ICU requirement are very low in patients whose COVID-GRAM score is defined as low-and medium risk ( Figure 3 , Table 3 ). In the patients defined as high risk, those without mortality or ICU outcome were predominated (63.9%-74.4%). The COVID-GRAM score classified only three patients at low risk in our study population. These three patients did not require ICU, and their outcome was not death. Therefore, the sensitivity of the COVID-GRAM score was found to be 100% ( Figure 3 , Table 3 ). For our study population, we can say that the COVID-GRAM score is insufficient for identifying low-risk patients. One of the reasons for this could be the high median age (67) of our study population. This is because in the COVID-GRAM score calculation-as can be seen from the COVID-GRAM risk calculation formula-age is contributed to the score by multiplying with a coefficient. As can be seen in Table 3 and in Figure 3B ,E, ISARIC-4C score points below 8 have very low mortality or ICU outcome. The fact that only 4 (1.6%) out of 251 patients defined as low and medium risk by the ISARIC-4C score who were hospitalised ICU from the ED, shows the success of ISARIC-4C in determining low risk ( Figure 3 , Table 3 ).

ISARIC-4C did not show the same success in the patient population that it describes as high risk. The patients defined as high risk by ISARIC-4C, who were 35.5% of the total, are in the IU group in terms of ICU requirement. When the same rate is evaluated in terms of inhospital mortality, 53.5% of the patients are in the survivor group.

Using YJI in addition to sensitivity, specificity and AUC provides more reliable results when comparing the accuracy of the scoring systems. 28 In our study, for the mortality prediction of COVID-19

patients, the YJI of CURB-65, ISARIC-4C and COVID-GRAM scores were found to be 0.58, 0.27 and 0.01 respectively; for determining the ICU requirement, the YJI were found to be 0.64, 0.26 and 0.01 respectively (Table 3) .

CURB-65 is a most well-known, simple to use and widely validated scoring system that has proven its prognostic accuracy. The CURB-65 score had the highest predictive accuracy also in our study population.

The sample size of this single centre study was relatively small. In retrospective studies, the study population is formed by convenience sampling methods, so it does not represent the general population and may lead to selection bias. Therefore, more studies with a larger sample size are needed to confirm these results.

Among the scores evaluated in our study, the CURB-65 score had better performance than ISARIC-4C and COVID-GRAM scores in predicting in-hospital mortality and ICU requirement in COVID-19

patients. Instead of COVID-GRAM score (which is more complex to calculate) we suggest that the CURB-65 score (which has been validated and can be easily calculated) can also be used for COVID-19

patients. Based on the success of ISARIC-4C in the low-risk group,

we believe that using CURB-65 and ISARIC-4C scores together will positively affect the decision-making process of clinicians.

The authors thank the employees of Kartal Dr Lütfi Kırdar City

Hospital for their dedication during the pandemic process.

The authors declare no conflict of interest.

This study was approved by the ethics committee of Kartal Dr Lütfi

Kırdar City Hospital (Ethics Committee Ruling number: 514/196/24).

Data available on request due to privacy/ethical restrictions. 

Fair Allocation of Scarce Medical Resources in the Time of Covid-19

World Health Organization. Naming the coronavirus disease (COVID-19) and the virus that causes it

Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study

Covid-19 in critically ill patients in the Seattle region-case series

Baseline characteristics and outcomes of 1591 patients infected with SARS-CoV-2 admitted to ICUs of the Lombardy Region

Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan, China

Performances of prognostic scoring systems in patients with healthcare-associated pneumonia

Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study

Assessment of risk scores in Covid-19

Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score

Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19

COVID-19 (SARS-CoV2 ENFEKSİYONU) REHBERİ. Erişim (Erişim Tarihi

Time course of lung changes on chest CT during recovery from 2019 novel coronavirus (COVID-19) pneumonia

The comparison of chest CT and RT-PCR during the diagnosis of COVID-19

BTS guidelines for the management of community acquired pneumonia in adults: update

Infectious Diseases Society of America/American Thoracic Society Consensus Guidelines on the management of community-acquired pneumonia in adults

Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach

Applicability of the CURB-65 pneumonia severity score for outpatient treatment of COVID-19

Simple Triage Scoring System (STSS) and Confusion, Uremia, Respiratory rate, Blood pressure and age ≥ 65 (CURB-65) score among patients with COVID-19 pneumonia in an emergency department triage setting: a retrospective study

CURB-65 may serve as a useful prognostic marker in COVID-19 patients within Wuhan, China: a retrospective cohort study

Comparison of CRB-65 and quick sepsisrelated organ failure assessment for predicting the need for intensive respiratory or vasopressor support in patients with COVID-19

Severity scores in COVID-19 pneumonia: a multicenter, retrospective, cohort study

Comparison of severity scores for COVID-19 patients with pneumonia: a retrospective study

Predicting critical illness on initial diagnosis of COVID-19: development and validation of the PRIORITY model for outpatient applicability

Predicting in-hospital mortality in COVID-19 older patients with specifically developed scores

Validation of a risk score to predict patients at risk of critical illness with COVID-19

Clinical risk prediction scores in coronavirus disease 2019: beware of low validity and clinical utility

Estimation of the Youden index and its associated cutoff point

How to cite this article: Doğanay F, Ak R. Performance of the CURB-65, ISARIC-4C and COVID-GRAM scores in terms of severity for COVID-19 patients