key: cord-0928574-a42v9oby
authors: Elazab, Ahmed; Elfattah, Mohamed Abd; Zhang, Yuexin
title: Novel multi-site graph convolutional network with supervision mechanism for COVID-19 diagnosis from X-ray radiographs
date: 2021-11-16
journal: Appl Soft Comput
DOI: 10.1016/j.asoc.2021.108041
sha: dfa108eb771d84c8bcd63e8fe2ae2b3dac146265
doc_id: 928574
cord_uid: a42v9oby

The novel Coronavirus disease 2019 (COVID-2019) has become a global pandemic and affected almost all aspects of our daily life. The total number of positive COVID-2019 cases has exponentially increased in the last few months due to the easy transmissibility of the virus. It can be detected using the nucleic acid test or the antibodies blood test which are not always available and take several hours to get the results. Therefore, researchers proposed computer-aided diagnosis systems using the state-of-the-art artificial intelligence techniques to learn imaging biomarkers from chest computed tomography and X-ray radiographs to effectively diagnose COVID-19. However, previous methods either adopted transfer learning from a pre-trained model on natural images or trained on limited datasets. Either cases may lead to accuracy deficiency or overfitting. In addition, feature space suffers from noise and outliers when collecting X-ray images from multiple datasets. In this paper, we overcome the previous limitation by firstly collecting a large-scale X-ray dataset from multiple resources. Our dataset includes 11,312 images collected from 10 different data repositories. To alleviate the effect of the noise, we suppress it in the feature space of our new dataset. Secondly, we introduce a supervision mechanism and combine it with the VGG-16 network to consider the differences between the COVID-19 and healthy cases in the feature space. Thirdly, we propose a multi-site (center) COVID-19 graph convolutional network (GCN) that exploits dataset information, the status of training samples, and initial scores to effectively classify the disease status. Extensive experiments using different convolutional neural network-based methods with and without the supervision mechanism and different classifiers are performed. Results demonstrate the effectiveness of the proposed supervision mechanism in all models and superior performance with the proposed GCN.

VGG-16 network to consider the differences between the COVID-19 and healthy cases in the feature space. 23 Thirdly, we propose a multi-site (center) COVID-19 graph convolutional network (GCN) that exploits 24 dataset information, the status of training samples, and initial scores to effectively classify the disease of the virus as they cannot be easily detected [5] . Although research institutions and pharmaceutical entities 43 have already developed some vaccines that could help combating COVID-19, the effectiveness of such 44 vaccines is still arguable. Therefore, early detection and diagnosis of COVID-19 remains a paramount task 45 in disease prevention and control. 46 The COVID-19 consists of single-stranded ribonucleic acid associated with 4 main structural proteins 47 (Spike, Envelope, Membrane, and Nucleocapsid) which can be detected using either the nucleic acid test 48 or the antibodies present in subject's blood [6] . Therefore, it was recommended by Chinese government to 49 confirm the diagnosis of COVID-19 using the reverse transcription polymerase chain reaction (RT-PCR) 50 [7]. However, the RT-PCR test kits availability is challenging in many regions in low-and middle-income 51 countries that leads to difficulties in samples collection and transportation. In addition, lab tests also suffer 52 from high false-negative cases that likely happen during samples' preparation and quality control [8] . 53 Furthermore, the RT-PCR process takes relatively long time (4-6 hours) to get results and may not be always 54 efficient [9] . Hence, many researchers tried to introduce fast, accurate, and low-cost methods for COVID- 55 19 diagnosis. 56 To this end, many researchers utilized the findings from medical imaging techniques (X-ray and 57 computed tomography (CT)) to early diagnose the suspected COVID-19 cases and other lung diseases as 58 well [10] . 

In this work, we collected a total number of 11,312 X-ray radiographs from 10 multi-center datasets. 171 In our dataset, we have 5656 and 5656 X-ray images for COVID-19 infected and heathy cases, respectively. 172 The detailed information of multi-center datasets is given in overview of whole COVID-19 diagnosis framework is shown in Fig. 2 . Note that, the words "multi-center" 187 and "multi-site" are used interchangeably throughout this paper. the edge weight between two subjects in the same group is calculated using the following formula: where represents the identity matrix whilst is the diagonal degree matrix. 275 After performing the spectral convolution operation, the adjacency matrix is estimated using 276 ∑ where is the polynomial order and it is used to control the filter effect (see its effect in 277 Section 5.B). 278 Finally, the proposed architecture combines two graph convolutional layers that use the ReLU as an 279 activation function (See Fig. 2(c) ). Following the two graph convolutional layers, a Softmax function is 280 used to output probability distribution for the final diagnosis of the disease. 

All experiments of this paper were conducted on a machine with Intel(R) CPU i7-8700 at 3.19 GHz, 284 GPU NVIDIA TITAN Xp, 128G of RAM, using Keras deep learning library. In our VGG network, we used 285 the following parameters. Learning rate was set to 1e -4 , Adam optimizer was adopted for optimization, and 286 the number of epochs was 200. In addition, our dataset was divided to 80% for training (in which 10% were 287 used for validation) and the remaining 20% for testing. The total running time is composed of the training 288 of the three phases of our framework; the VGG feature extraction, the supervision mechanism, the GCN 289 diagnosis (Fig. 2 (a)-(c) ). The training time for these phases was, respectively, 2.27 hr, 1.84 hr, and 0.4 hr 290 (totally 4.12 hr).

In this work, we evaluated the performance of our methods using several metrics. Namely, we measured 292 the sensitivity (Sens), specificity (Spec), accuracy (ACC), and area under receiver operating characteristic 293 (ROC) curve which are defined as: 

In this paper, we used several deep learning-based methods to perform feature extraction. These In Table 4 , we summarize the classification results for our binary task (COVID-19 vs. Healthy) using 314 different feature extraction methods (with and without the supervision mechanism) and classifiers. It can 315 be noticed that, the proposed supervision mechanism effectively improves the performance of all classifiers. 316 In particular, the proposed GCN with the supervision mechanism on the VGG network achieves 96.4%, 317 96.6%, 96.2%, and 98.7% for ACC, Sens, Spec, and AUC, respectively. Best results are given in boldfaces 318 for every classifier. In addition, we compare the performance of our method with the state-of-the-art 319 methods for COVID-19 diagnosis. Comparisons are given in Table 5 . From this Table, it is obvious that the 320 proposed method is not only trained on the biggest dataset but also it achieves the best performance in 321 COVID-19 diagnosis. Note that, although the dataset used in Wang et al. [44] is apparently the biggest 322 dataset, the major part of this dataset is generated one, not real X-ray images. Similarly, Luz et al. [51] used 323 a quite big dataset of 13,569 subjects (but only from 3 centers), yet, our performance has about 2.5% higher 324 accuracy than it. Despite our proposed method achieves higher accuracy than the methods in Table 5, there   325 are other methods in literature that attained better accuracy than ours e.g., [35, 36, 40, 54, 55, 57, 58] . 326 Nevertheless, these methods were tested on a relatively small datasets with limited number of centers which 327 may not be appropriate and thus hinder their clinical significance, without further investigations. It is 328 noteworthy that, in our dataset, we maintain the class balance by choosing equal number of the healthy and 329 the COVID-19 subjects (see Table 2 ) which not only neutralizes the feature learning process but also further 330 promotes the reliability of the proposed method performance. 331 In addition, in Fig. 3 , we depict the performance of the proposed method for the both accuracy and the 332 loss function during the training phase. It is shown that, the training and validation accuracies of our method 333 reaches 99.01% and 96.4%, respectively, at epoch 64 (Fig. 3a) . Similarly, our loss function (categorical 334 crossentropy) has a stable behavior with almost same generalization gap around epoch 98 (Fig. 3b) . The 335 stability of the accuracy and the loss function during training process is mainly due to the effectiveness of 336 our proposed supervision mechanism which enhances the feature learning ability in our model. 337 We further draw the ROC curves to demonstrate the effectiveness of the proposed supervision 338 mechanism on the performance of different classifiers as shown in Fig. 4 . It is clear that the classification 339 performance is improved using our supervision mechanism, particularly with the proposed GCN. mechanism can enhance the diagnosis performance not only of our GCN but also for other methods as well.

In this Section, we discuss the main advantages of proposed method and the current limitations in our 349 study. Besides, we also investigate the effect of the supervision mechanism on the feature extraction, the 350 polynomial order, and the number of features as well. respectively, when the supervision mechanism is adopted. 360 We also demonstrate how the top 12 features are affected when using the proposed supervision 361 mechanism in Fig. 5 . In this figure, features of COVID-19 (yellow and red) and healthy (blue and green) 362 subjects with and without supervision are plotted. It can be noticed that, there is a great difference between 363 COVID-19 and healthy subjects when supervision mechanism is adopted as compared to no supervision. 364 This can ultimately ease and boost the classification task by using the underlying classifier. 365 Moreover, we also evaluate the effect of weightings of the supervision mechanism on the accuracy of 366 COVID-19 diagnosis. The eminent performance of our proposed method can help in combating the COVID-19 pandemic by 416 providing an early diagnosis tool from chest X-ray radiographs, particularly in the low-income and isolated 417 regions where RT-PCR test kits are not available. After further investigations, we believe that the proposed 418 method can be adopted in the healthcare systems in different ways, especially with the development of the 419 internet of things. One way to achieve this is incorporating our method with the screening system of the 420 chest X-ray scanner to provide the radiologists/technicians with preliminary diagnosis. Another possible 421 way is utilizing the proposed method (together with other patient's clinical data) in remote computer-aided 422 diagnosis system on a cloud system platform. Finally, it might also be useful to develop a light-weight 423 application of the proposed method to be applicable on hand-held devices. 424 In our future work, we will evaluate the performance of the proposed method on multimodal data (i.e. 425 CT images and X-ray radiographs) towards learning complementary features. In addition, it will be 426 interesting to measure the severity of the COVID-19 on the chest tissues to evaluate the risk of the infection. 427 Last but not least, it will be highly desirable to learn the spread of the infection in the tissue. 

Last Access

Mathematical models to guide pandemic response

A novel coronavirus from patients with pneumonia in China

Clinical Course and Molecular Viral Shedding Among Asymptomatic and Symptomatic 447

Patients With SARS-CoV-2 Infection in a Community Treatment Center in the Republic of Korea

Genotype and phenotype of COVID-19: Their roles in 450 pathogenesis

Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in 452 China: a report of 1014 cases

A Rapid, Accurate and Machine-Agnostic Segmentation and Quantification Method 454 for CT-Based COVID-19 Diagnosis

The incubation period of coronavirus disease 2019 (COVID-19) from publicly 457 reported confirmed cases: estimation and application

Lung nodule classification using deep feature fusion in chest 460 radiography

Correlation Between Low-Dose Chest Computed Tomography and RT-PCR Results for 462 the Diagnosis of COVID-19: A Report of 27,824 Cases in Tehran, Iran

Systematic review of artificial intelligence techniques in the detection and 465 classification of COVID-19 medical images in terms of evaluation and benchmarking: Taxonomy analysis, challenges, future solutions and methodological aspects

A review of deep learning in medical imaging: Image traits, technology trends, case 469 studies with progress highlights, and future promises

Artificial intelligence and COVID-19: deep learning approaches for diagnosis and 471 treatment

Prediction models for diagnosis and prognosis of covid-19 infection: systematic 473 review and critical appraisal

Applications of machine learning and artificial 475 intelligence for Covid-19 (SARS-CoV-2) pandemic: A review

Review of artificial intelligence techniques in imaging data acquisition, segmentation and 478 diagnosis for covid-19

Early survey 480 with bibliometric analysis on machine learning approaches in controlling COVID-19 outbreaks

Imagenet classification with deep convolutional neural 483 networks

Deep residual learning for image recognition

Very deep convolutional networks for large-scale image recognition

Covid-19: automatic detection from x-ray images utilizing 489 transfer learning with convolutional neural networks

Deep Convolutional Neural Network with Transfer 492 Learning for Detecting Pneumonia on Chest X-Rays

CovXNet: A multi-dilation convolutional neural 495 network for automatic COVID-19 and other pneumonia detection from chest X-ray images with 496 transferable multi-receptive feature optimization

COVID-19 from chest X-ray images using deep transfer learning

Deep transfer learning 502 artificial intelligence accurately stages COVID-19 lung disease severity on portable chest radiographs

Extracting possibly representative COVID-505

Biomarkers from X-Ray images with Deep Learning approach and image data related to Pulmonary 506

Explainable deep learning for pulmonary 508 disease and coronavirus COVID-19 detection from X-rays

Residual Attention U-Net for Automated Multi-Class Segmentation 511 of COVID-19 Chest CT Images

PDCOVIDNet: a parallel-dilated convolutional 513 neural network architecture for detecting COVID-19 from chest X-ray images

Early diagnosis of COVID-19-affected patients based on X-ray and computed 516 tomography images using deep learning algorithm

Attention u-net based adversarial architectures for chest x-ray lung 518 segmentation

Covid-cxnet: Detecting covid-19 in frontal chest x-ray 520 images using deep learning

Covidx-net: A framework of deep learning 522 classifiers to diagnose covid-19 in x-ray images

A combined deep cnn-lstm network for the detection of novel 524 coronavirus (covid-19) using X-ray images

A deep learning approach to detect Covid-19 527 coronavirus with X-Ray images

Coronet: A deep neural network for detection and diagnosis 529 of COVID-19 from chest x-ray images

Automatic detection of coronavirus disease (covid-19) using x-ray 532 images and deep convolutional neural networks

Deep learning covid-19 features on cxr using limited training data sets

Automated 536 detection of COVID-19 cases using deep neural networks with X-ray images

Application of Deep 539 Learning for Fast Detection of COVID-19 in X-Rays using nCOVnet

Weakly Labeled Data Augmentation for Deep Learning: A Study on 542 COVID-19 Detection in Chest X-Rays

COVIDiagnosis-Net: Deep Bayes-SqueezeNet based Diagnostic of the 544

COVID-19) from X-Ray Images

COVID-Net: A Tailored Deep Convolutional Neural Network Design for 546 Detection of COVID-19 Cases from Chest X-Ray Images

Chest CT for typical 2019-nCoV 548 pneumonia: relationship to negative RT-PCR testing

Deep learning-based decision-tree classifier for COVID-19 diagnosis from chest X-550 ray imaging

COVID-19 detection and disease progression visualization: Deep learning on 552 chest X-rays for classification and coarse localization

Covid-19 screening on chest x-ray images using deep 554 learning based anomaly detection

COVID-MobileXpert: On-Device COVID-19 Screening using Snapshots of 556

Convolutional Neural Network for COVID-19 Outbreak Screening Using Chest X-rays

Towards an effective and efficient 561 deep learning model for covid-19 patterns detection in x-ray images

Augmentation Using Auxiliary Classifier GAN for Improved Covid-19 Detection

Classification of COVID-19 in chest X-ray images 566 using DeTraC deep convolutional neural network

CoVNet-19: A Deep Learning model for the detection and analysis 569 of COVID-19 patients

DeepCoroNet: A deep LSTM approach for automated detection of COVID-19 cases from 571 chest X-ray images

Toward Smart Lockdown: A Novel Approach for COVID-573

Hotspots Prediction Using a Deep Hybrid Neural Network

CNN-based transfer learning-BiLSTM 575 network: A novel approach for COVID-19 infection detection

One-shot Cluster-Based 578 Approach for the Detection of COVID-19 from Chest X-ray Images

Detection of coronavirus disease (COVID-19) 581 based on deep features and support vector machine

Deep convolutional neural networks to diagnose covid-19 and other 583 pneumonia diseases from posteroanterior chest x-rays

Gene selection for cancer classification using support 585 vector machines

Faster r-cnn: Towards real-time object detection with region 587 proposal networks

Attribute-aware attention model for fine-grained representation 589 learning

Classification of Mild Cognitive Impairment Based on a Combined 592

High-Order Network and Graph Convolutional Network

Disease prediction using graph convolutional networks: application to autism 594 spectrum disorder and Alzheimer's disease

Multilayer perceptron, fuzzy sets, classifiaction

Classification and regression by randomForest

Stochastic gradient boosted distributed decision trees