key: cord-0869961-xm92rbdp
authors: Luo, Xufei; Liu, Yunlan; Ren, Mengjuan; Zhang, Xianzhuo; Janne, Estill; Lv, Meng; Wang, Qi; Song, Yang; Mathew, Joseph L.; Ahn, Hyeong Sik; Lee, Myeong Soo; Chen, Yaolong
title: Consistency of recommendations and methodological quality of guidelines for the diagnosis and treatment of COVID‐19
date: 2021-02-09
journal: J Evid Based Med
DOI: 10.1111/jebm.12419
sha: 08a37d87e8080bbac7e6a6031986d78abb5d1142
doc_id: 869961
cord_uid: xm92rbdp

OBJECTIVE: Since the beginning of the COVID‐19 epidemic, a large number of guidelines on diagnosis and treatment of COVID‐19 have been developed, but the quality of those guidelines and the consistency of recommendations are unclear. The objective of this study is to evaluate the quality of the diagnosis and treatment guidelines on COVID‐19 and analyze the consistency of the recommendations of these guidelines. METHODS: We searched for guidelines on diagnosis and/or treatment of COVID‐19 through PubMed, CBM, CNKI, and WanFang Data, from January 1, 2020 to August 31, 2020. In addition, we also searched official websites of the US CDC, European CDC and WHO, and some guideline collection databases. We included diagnosis and/or treatment guidelines for COVID‐19, including rapid advice guidelines and interim guidelines. Two trained researchers independently extracted data and four trained researchers evaluated the quality of the guidelines using the AGREE II instruments. We extracted information on the basic characteristics of the guidelines, guideline development process, and the recommendations. We described the consistency of the direction of recommendations for treatment and diagnosis of COVID‐19 across the included guidelines. RESULTS: A total of 37 guidelines were included. Most included guidelines were assessed as low quality, with only one of the six domains of AGREE II (clarity of presentation) having a mean score above 50%. The mean scores of three domains (stakeholder involvement, the rigor of development and applicability) were all below 30%. The recommendations on diagnosis and treatment were to some extent consistent between the included guidelines. Computed tomography (CT), X‐rays, lung ultrasound, RT‐PCR, and routine blood tests were the most commonly recommended methods for COVID‐19 diagnosis. Thirty guidelines were on the treatment of COVID‐19. The recommended forms of treatment included supportive care, antiviral therapy, glucocorticoid therapy, antibiotics, immunoglobulin, extracorporeal membrane oxygenation (ECMO), convalescent plasma, and psychotherapy. CONCLUSIONS: The methodological quality of currently available diagnosis and treatment guidelines for COVID‐19 is low. The diagnosis and treatment recommendations between the included guidelines are highly consistent. The main diagnostic methods for COVID‐19 are RT‐PCR and CT, with ultrasound as a potential diagnostic tool. As there is no effective treatment against COVID‐19 yet, supportive therapy is at the moment the most important treatment option.

To our knowledge no study has yet systematically compared the recommendations of COVID-19 diagnosis and treatment guidelines, for quality and consistency. Therefore, we conducted this study to evaluate the quality of COVID-19 diagnosis and treatment guidelines developed exclusively for COVID-19, and compare the similarities and differences in the diagnostic and treatment recommendations of these guidelines.

We conducted a retrospective cross-sectional analysis of diagnosis and treatment guidelines on COVID-19. Before the study began, the reviewers familiarized themselves with the methodological appraisal tool for guidelines via AGREE II 9 (Appraisal of Guidelines, for Research, and Evaluation). 

We included diagnosis and/or treatment guidelines for COVID-19, including rapid advice guidelines and interim guidelines. A diagnostic and/or treatment guideline was defined as a guideline that contains diagnostic and/or treatment elements in their recommendations.

If the guideline was published in both English language and Chinese language, we only included the English-language version. If there were multiple versions of the guidelines, we only included the latest one.

We excluded the following types of documents: (1) 

We used Endnote X9 for literature storage and screening. Two trained researchers (YL, ML) screened independently first by the titles and abstracts. Two researchers (YL, ML) independently screened records from the official websites manually. Discrepancies were solved by a third researcher (XL). Full text reviewing was done by one researcher (XL).

For each eligible diagnosis and treatment guideline, we extracted the following data: (1) basic characteristics of the guidelines (eg, title, publication date, country/setting, type of guideline); (2) characteristics of the developer and development process of the guideline (eg, the developer institution or journals, whether a systematic review was conducted, whether the evidence was formally graded, system of evidence grading); and (3) recommendations (diagnosis and treatment details). Two reviewers (XL, ML) independently extracted the data, which were then checked by a third person (YL). Disagreements were solved through discussion or with the help of a fourth author (YC).

We used the AGREE II tool to evaluate the quality of the included guidelines. The tool consists of 23 key items divided into six domains.

Each item is assigned a score from 1 (strongly disagree) to 7 (strongly agree). The score for each domain was obtained by adding up the scores of all the items in the domain (of each assessor) and normalizing them (score obtained -minimum possible score)/(maximum possible scoreminimum possible score). Each guideline was independently assessed by four reviewers. We calculated the mean scores for each domain across all guidelines, as well as for guidelines developed in each country, territory or area separately.

We conducted descriptive analyses of the general characteristics of the 

A total of 2018 articles were identified, and 37 records 4,5, were finally included after successive screening, including 32 guidelines 4, 5, 11, [14] [15] [16] [17] [18] [19] [20] 22, 23, [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] (86.5%) published in English language and five 10,12,13,21,24 (13.5%) in Chinese language (Fig. 1 ).

Of the 37 guidelines included, ten 4 25 India, 16 Italy, 19 Mexico, 31 North America (USA and Canada), 26 Poland, 29 and Spain. 27 These guidelines were published between January 30 and August 27, 2020.

The ICC between the four reviewers for AGREE II assessment in the study was 0.925 (95%CI, 0.907-0.938). The quality of the most included guidelines was in general low, and the mean score over all guidelines was above 50% in only one domain (clarity of presentation).

Three domains (stakeholder involvement, rigor of development and applicability) had a mean score below 30% over the guidelines. The guidelines developed by multicountry international panels had the highest scores, whereas the guidelines from Poland, Germany, and

Australia had the lowest scores ( Figure 2 ). recommendations in using antiviral treatment were also inconsistent.

Twenty guidelines 4, 10, 11, [14] [15] [16] 18, 22, 23, [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] 41 (54.1%) addressed the use of glucocorticoids for patients with COVID-19.

Nine 4, 14, 16, 22, 28, 30, 34, 36, 41 (24.3%) of these did not recommend routinely using the glucocorticoids or except in the context of clinical trials. The remaining nine guidelines 10, 11, 15, 18, 27, 29, 32, 33, 35 conditionally recommended the use of glucocorticoids in specific conditions, such as for critically ill patients with COVID-19. The glucocorticoids often mentioned in these guidelines included methylprednisolone and dexamethasone.

The use of antibiotics during COVID-19 treatment was also a common topic. We found a total of 17 guidelines 4, 10, 11, [13] [14] [15] [16] 22, 24, 27, [29] [30] [31] [33] [34] [35] 43 (45.9%), either recommending or specifically not recommending the use of antibiotics in patients with COVID-19. Most of these guidelines 10, 11, [13] [14] [15] [16] 22, 25, 27, 29, 31, 34, 35, 43 (n = 14, 37.8%) conditionally recommended starting an empiric broadspectrum antibiotic therapy when patients were suspected to have a bacterial superinfection or when their condition got worse. However, a prophylactic antibiotic therapy or routine treatment with antibiotics was not recommended in any guideline.

A total of six guidelines 15 

We evaluated the quality of 37 diagnosis and treatment guidelines 4,5,10-44 for COVID-19 and analyzed the consistency of their recommendations. In general, we found these guidelines to be of low quality. Their recommendations for treatment and diagnosis were somewhat consistent, but there were differences in the recommendations considering some specific populations and conditions.

Our results suggested that the quality of diagnosis and treatment guidelines for COVID-19 was low, especially regarding stakeholder involvement, rigor of development, and applicability. This was generally consistent with the findings of two earlier studies. 6, 7 There are three potential reasons for this. First, in the first eight months of the COVID-19 epidemic, most guidelines were rapid advice guidelines or interim guidance, and the low quality may be a result of the urgency and lack of time. However, the quality of the guidelines has increased over time. Second, some of the included guidelines were published on websites only, reported with insufficient details resulting in low quality. Third, some of the AGREE II items (eg, item 7) are not necessarily applicable to the evaluation of the rapid advice guidelines, which may also have contributed to the low quality of some of the guidelines. In addition, the lack of sufficient and direct evidence to support the guidelines in the early stages of the COVID-19 epidemic also contributes to the poor quality, the quality of evidence for guidelines based need to be improved. 45 To improve the quality of guidelines developed in such urgent situations, guideline developers should learn how to develop rapid advice guidelines in public health emergencies. Guideline users should also learn how to quickly assess the quality of public health emergencies guidelines in order to better guide public health and clinical practice. 52 had also been registered on ClinicalTrials.gov platform. On the one hand, these trials and reviews can provide evidence for future guidelines, but on the other hand, low-quality or duplicate studies can lead to wasted research resources. 53 In the absence of effective treatment drugs, a vaccine is likely to be the most effective measure to end the epidemic. In the meantime, preventing and controlling the spread of COVID-19 is paramount, especially in low-and middle-income countries and countries with fragile health systems.

To our knowledge, this is the first study to analyze and evaluate the diagnosis and treatment guidelines for SARS-CoV-2 infection.

We used a systematic searching method, summarized recommendations, evaluated the quality of the guidelines, and examined the consistency of recommendations. However, this study also has several limitations. First, the searching time was up to August 31, 2020, and some guidelines that are still under development or have not yet been indexed may have been omitted. Second, we only analyzed and evaluated the COVID-19 diagnosis and treatment guidelines. However, guidelines for COVID-19 in combination with other diseases can also inform clinical decision-making. Third, due to the diversity of contents in the guidelines, we were unable to extract and analyze all recommendations.

In conclusion, the current diagnosis and treatment guidelines 

World Health Organization. Coronavirus disease (COVID-19) pandemic

Clinical management of severe acute respiratory infection (SARI) when COVID-19 disease is suspected: interim guidance

treatment of patients with nonsevere and severe coronavirus disease 2019: an evidence-based guideline

Clinical management of COVID-19 interim guidance

Interim Guidelines for management of confirmed Coronavirus Disease (COVID-19) clinical care guidance

Scope, quality, and inclusivity of clinical guidelines produced early in the COVID-19 pandemic: rapid review

COVID-19 evidence and recommendations working group. A quality evaluation of guidelines on five different viruses causing public health emergencies of international concern

Consistency of recommendations from clinical practice guidelines for the management of critically ill COVID-19 patients

AGREE II: advancing guideline development, reporting and evaluation in health care

Rapid advice guideline for the diagnosis and treatment of the novel coronavirus pneumonia infections

A rapid advice guideline for the diagnosis and treatment of 2019 novel coronavirus (2019-nCoV) infected pneumonia (standard version)

Committee of the Infectious Diseases Radiology Group of Chinese Society of Radiology, et al. Guideline for imaging diagnosis of novel coronavirus (2019-nCoV) infected pneumonia

Children's Hospital of Fudan University. Rapid screening and practice guidelines for suspected and confirmed cases of novel coronavirus infection/pneumonia in children

Corticosteroid therapy for critically ill patients with COVID-19: a structured summary of a study protocol for a prospective meta-analysis of randomized trials

Diagnosis and clinical management of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection: an operational recommendation of Peking Union Medical College Hospital (V2.0)

Government of India Ministry of Health & Family Welfare, Directorate General of Health Services (EMR Division)

Guidelines for laboratory diagnosis of coronavirus disease 2019 (COVID-19) in Korea

Management of critically ill adults with COVID-19

The Italian coronavirus disease 2019 outbreak: recommendations from clinical practice

Physiotherapy management for COVID-19 in the acute hospital setting: clinical practice recommendations

Guidelines for imaging diagnosis of novel coronavirus pneumonia (second edition 2020, shortened version)

Clinical management of patients with moderate to severe COVID-19-interim guidance

Corticosteroid guidance for pregnancy during COVID-19 pandemic

Chinese Medical Association. Radiological diagnosis of COVID-19: expert recommendation from the Chinese Society of Radiology

German recommendations for critically ill patients with COVID 19

Multicenter initial guidance on use of antivirals for children with COVID-19/SARS-CoV-2

Grupo de trabajo de la Asociación Española de Pediatría para el brote de infección por Coronavirus, colaboradores con el Ministerio de Sanidad. Recommendations on the clinical management of the COVID-19 infection by the «new coronavirus» SARS-CoV2. Spanish Paediatric Association working group

Infectious Diseases Society of America Guidelines on the treatment and management of patients with coronavirus disease 2019 (COVID-19)

Management of SARS-CoV-2 infection: recommendations of the Polish Association of Epidemiologists and Infectiologists as of March 31

Rapid advice guidelines for management of children with COVID-19

Recommendations for the management of critically ill adult patients with COVID-19

Treatment of patients with nonsevere and severe coronavirus disease 2019: an evidence-based guideline

Interim guidelines on antiviral therapy for COVID-19

Diagnosis and treatment recommendations for pediatric respiratory infection caused by the 2019 novel coronavirus

Recommendations for the prevention and treatment of the novel coronavirus pneumonia in the elderly in China

Infectious Diseases Society of America Guidelines on the Diagnosis of COVID-19

COVID-19 PICU guidelines: for high-and limited-resource settings

Public Health Laboratory Network Guidance for serological testing in COVID-19

Clinical management of coronavirus disease 2019 (COVID-19) in pregnancy: recommendations of WAPM-World Association of Perinatal Medicine

Use of chest imaging in the diagnosis and management of COVID-19: a WHO rapid advice guide

Remdesivir for severe COVID-19: a clinical practice guideline

PHLN guidance on laboratory testing for SARS-CoV-2 (the virus that causes COVID-19)

Clinical management of patients with COVID-19: second interim guidance

Coronavirus disease 2019 (COVID-19) treatment guidelines

The quality of evidence in Chinese metaanalyses needs to be improved

Application of lung ultrasound during the COVID-19 Pandemic: A Narrative Review

Lung ultrasound in the diagnosis of COVID-19 infection-a case series and review of the literature

Therapeutic options for the treatment of coronavirus disease (COVID-19)

Systematic review and meta-analysis of the effectiveness and safety of hydroxychloroquine in treating COVID-19 patients

Drug treatments for COVID-19: living systematic review and network meta-analysis

Lopinavir-ritonavir for COVID-19: a living systematic review

Systematic review of the registered clinical trials for coronavirus disease 2019 (COVID-19)

Avoidable waste of research on coronavirus disease 2019 (COVID-19)

Consistency of recommendations and methodological quality of guidelines for the diagnosis and treatment of COVID-19

The authors declare no conflict of interest.

Xufei Luo https://orcid.org/0000-0003-0811-6326 Yaolong Chen https://orcid.org/0000-0002-8598-6504