key: cord-0232361-7bv056yw authors: Rahimzadeh, Mohammad; Attar, Abolfazl title: A New Modified Deep Convolutional Neural Network for Detecting COVID-19 from X-ray Images date: 2020-04-17 journal: nan DOI: nan sha: 5dc2a2b45f53f6dab91657b8fd658ee8e406a462 doc_id: 232361 cord_uid: 7bv056yw COVID-19 has become a serious health problem all around the world. It is confirmed that this virus has taken over 126,607 lives until today. Since the beginning of its spreading, many Artificial Intelligence researchers developed systems and methods for predicting the virus's behavior or detecting the infection. One of the possible ways of determining the patient infection to COVID-19 is through analyzing the chest X-ray images. As there are a large number of patients in hospitals, it would be time-consuming and difficult to examine lots of X-ray images, so it can be very useful to develop an AI network that does this job automatically. In this paper, we have trained several deep convolutional networks with the introduced training techniques for classifying X-ray images into three classes: normal, pneumonia, and COVID-19, based on two open-source datasets. Unfortunately, most of the previous works on this subject have not shared their dataset, and we had to deal with few data on covid19 cases. Our data contains 180 X-ray images that belong to persons infected to COVID-19, so we tried to apply methods to achieve the best possible results. In this research, we introduce some training techniques that help the network learn better when we have few cases of COVID-19, and also we propose a neural network that is a concatenation of Xception and ResNet50V2 networks. This network achieved the best accuracy by utilizing multiple features extracted by two robust networks. In this paper, despite some other researches, we have tested our network on 11302 images to report the actual accuracy our network can achieve in real circumstances. The average accuracy of the proposed network for detecting COVID-19 cases is 99.56%, and the overall average accuracy for all classes is 91.4%. The pervasive spread of the coronavirus around the world has quarantined many people and crippled many industries, which has had a devastating effect on human life quality. So far, the coronavirus has infected at least 1.9 million people and killed around 126,607 patients [14] . Due to the high transmissibility of coronavirus, the detection of coronavirus disease-19 plays an essential role in how to control and plan to prevent COVID- 19 [24] . On the other hand, the lack of detective tools and the limitations in medical equipment and experts has slowed down the disease detection. As a result, it increases the number of patients and casualties. If the disease is detected quickly, its prevalence and the number of casualties due to COVID-19 disease will decrease. In recent years, computer vision and Deep Learning have been used to detect many different diseases and lesions in the body automatically [12] . Some examples are: Detection of tumor types and volume in lungs, breast, head and brain [2, 10] , state-of-the-art bone suppression in x-rays, diabetic retinopathy classification, prostate segmentation, skin lesion classification, analysis of the myocardium in coronary CT angiography [27] ,nodule classification [12] , sperm detection and tracking for male fertility analysis [16] , etc. Given that chest CT scan and X-ray images analysis is one of the ways for diagnosing COVID-19, the use of computer vision and Deep Learning can play a beneficial role in the diagnosis of this disease. Since the disease became widespread, many researchers have used machine vision and Deep Learning methods and obtained good results [1, 26] . The paper is organized as follows: In Section 2, the previous works related to our work are presented. In Section 3, we describe the used Dataset, and we explain the utilized neural networks and training techniques. In Section 4, we have presented the experimental results, and then the paper is concluded in Section 5. In Section 6, we presented the trained networks and the codes used in this research. In [1, 26] , statistical analysis of CT scans was performed by several specialists and diagnosticians, who classified the suspects into several classes for diagnosis and treatment. Because of the superiority of computer vision and Deep Learning in the analysis of medical images, after the reliability of CT scans of the chest for COVID-19 detection, the researchers used these tools to diagnose COVID-19. Immediately, artificial intelligence began to detect the disease and measure the rate of infection and damage to the lungs using CT scans and the course of the disease, with promising results [7] . IN [23] , they have used an innovative CNN to classify and predict COVID-19 using lung CT scans. [7] has used Deep Learning to detect COVID-19 and segment the lung masses caused by the coronavirus using 2D and 3D images. COVID-Net uses a lightweight residual projectionexpansion-projection-extension (PEPX) design pattern to investigate quantitative analysis and qualitative analysis [22] .In another research, pre-trained ResNet50, InceptionV3, and Inception ResNetV2 models have used with transfer learning techniques to classify Chest X-ray images normal and COVID-19 classes [15] . In [11] , they present COVNet for predict COVID-19 from CT scans that have segmented using U-net [17] . Another research has combined the Human-In-The-Loop(HITL) strategy that involved a group of chest radiologists with deep learning-based methods to segment and measures infection in CT scans [18] . In [25] , they have tried to detect COVID-19 and Influenza-A-viral-pneumonia from their data; They have used classical ResNet-18 network structure to extract the features, and another Innovative CNN network uses these features by creating the location-attention oriented model to classify the data. We have used two open-source datasets in our work. The first dataset is taken from this GitHub repository (https: //github.com/ieee8023/covid-chestxray-dataset), which has been prepared by [5] . This dataset consists of X-ray and Ct scan images of patients infected to COVID-19, SARS, Streptococcus, ARDS, Pneumocystis, and other types of pneumonia from different patients. In this dataset, we only considered the x-ray images, and in total, there were 180 images from 118 cases with COVID-19 and 42 images from 25 cases with Pneumocystis, Streptococcus, and SARS that were considered as pneumonia. The second dataset was taken from (https://www.kaggle.com/c/rsna-pneumonia-detection-challenge), which contains 6012 cases with pneumonia, and 8851 normal cases. We combined these two datasets, and the details are listed in table 1. As it is stated, we only had 180 cases infected to COVID-19, which is almost a few numbers of data for a class. If we had combined lots of images from normal or pneumonia classes with few images of COVID-19 class, the network would become able to detect pneumonia and normal classes very well, but not the COVID-19 class. In that case, as there are many more images of pneumonia and normal classes than the COVID-19 class, the general accuracy would become very high, not the COVID-19 detection accuracy, which is not our goal because the main purpose here is to achieve good results in detecting COVID-19 cases and not detecting false COVID-19 cases. We decided to select 250 random cases of normal class and 234 random cases of pneumonia class along with the 149 COVID-19 cases for training. In total, we had 633 cases for the training set. We used another method for also improving the general detection accuracy. In this method, we selected eight different training sets with 633 cases that 149 of them were COVID-19 cases and 34 pneumonia cases from the first dataset that is common between all sets. Each set contained 250 normal and 200 pneumonia cases that are unique. Based on this categorizing, our training set than includes eight sets is made of 3783 images. By doing so, we show different normal and pneumonia cases to the network in each training phase along with the same COVID-19 cases. This makes the network better be able to distinguish the COVID-19 cases from the other ones, so the rate of false detected COVID-19 cases decreases. This condition also causes the network to learn other classes features better, so the general accuracy will increase. We allocated 8 phases for training, which in each training phase one train set was fed to the network for training during 100 epochs; in total, we trained the network for 800 epochs. For reporting more accurate results, we chose five folds for training, which in every fold the training set was made of 8 phases as it is mentioned. Some of the images of our dataset are shown in figure 1. We have trained ResNet50V2 [8] , Xception [3] , and a concatenation of Xception and ResNet50V2 neural networks based on the explained method. The concatenated neural network is designed by concatenating the extracted features of Xception and ResNet50V2 and then connecting the concatenated features to a convolutional layer that is connected to the classifier. This Neural Network has shown higher accuracy comparing to the others. As we have tested several networks in our project, the Xception [3] and ResNet50V2 [8] networks works almost better than other ones in extracting deep features. By concatenating the output features of both networks, we helped the network to learn to classify the input image from both feature vectors, and this has resulted in better accuracy. The architecture of the concatenated network has been depicted in figure 2 , and the training parameters have been described in table2. We implemented the neural networks with Keras[4] library on a Tesla P100 GPU and 25GB RAM that were provided by Google Colaboratory Notebooks. We validated our data on 31 cases of COVID-19, 4420 cases of pneumonia, and 6851 normal cases. The reason our training data was less than validation data is that, because having few cases of COVID-19 among lots of normal and with pneumonia cases, we could not use lots of images of 2 other classes because it would cause the network not to learn COVID-19 features. We selected a part of the dataset for eight phases of training in each fold, as it is mentioned before, and we evaluated our network on the rest of the data so that the ultimate performance of our trained network be clear. It is noteworthy that we used transfer learning in training precess. For all the networks, we used the pre-trained ImageNet weights [6] at the beginning of the training and then resumed training based on the explained conditions on our dataset. The evaluation results of the neural networks are presented in figure 3 that shows the confusion matrixes of each network for fold one and three. Tables 3 and 4 show the details of our results. The metrics we reported for evaluating our network are described below: Accuracy (f or all the classes) = N umber of Correct Classif ied Images N umber of All Images Accuracy (f or each class) = T P + T N T P + F P + T N + F N (2) P recision = T P T P + F P In these equations, T P (true positive) is the number of correctly classified images of a class, F P (false positive) is the number of the wrong classified images of a class, F N (false negative) is the number of images of a class that have been detected as another class, and T N (true negative) is the number of images that do not belong to a class and won't be classified as that class. It can be understood from the confusion matrixes and the tables that the concatenated network performs better in detecting COVID-19 and not detecting false cases of COVID-19 and outputs better accuracy overall. The reason the precision of COVID-19 class is low is that in our work, despite some other researches that worked on detecting COVID-19 from X-ray images, we tested our neural nets on a massive number of images. Our test images were much more than our train images. As it is explained above, because we had only 31 cases of COVID-19 and 11271 cases from other two classes, the false positives of the COVID-19 class would become more than true positives, e.g., the concatenated network in the first fold, detected 26 cases correctly out of 31 COVID-19 cases and from 11271 other cases, only mistakenly detects 68 cases as COVID-19. If we had equal samples from COVID-19 class as the other classes, the precision would become a high value, but because having few COVID-19 cases and lots of other cases for validation, the precision would become a low value. We could have tested our network on a few cases like some of the other researches done recently, but we wanted to show the real performance of our network with few COVID-19 cases. As mentioned, mistakenly detecting 68 cases from 11271 cases to be infected to COVID-19 is not very much but not very well also, and we hope that by using much-provided data from patients infected, COVID-19, the detection accuracy will rise much more. In this paper, we presented a concatenated neural network based on Xception and ResNet50V2 networks for classifying the chest X-ray images into three categories of normal, pneumonia, and COVID-19. We used two open-source datasets that contained 180 and 6054 images from patients infected to COVID-19 and pneumonia, respectively, and 8851 images from normal people. Although we had a few images of COVID-19 class, we selected 633 images for training in each phase, so that our network learn COVID-19 class characteristics better, not only the other two classes. We considered eight phases for the training process, which in every phase the images from normal and pneumonia classes were different so that the network can distinguish COVID-19 from other classes better. In total out training set included 3783 images, and The rest of the images were chosen for evaluating the network. We tried to test our model on a large number of images so that our real achieved accuracy be clear to everyone. We achieved an average accuracy of 99.56%, and 80.53% recall for the COVID-19 class, and the overall accuracy equal to 91.4% between five folds. We hope that our trained network that is available for the public comes with the help of medical diagnosis. We also hope that in the future, larger datasets from COVID-19 patients become available, and by using them, the accuracy of our network increases for good. Confusion Matrix for the concatenated network-fold1 In this GitHub profile (https://github.com/mr7495/covid19), we have shared the trained networks and all the used code in this paper. We hope our work be useful to help in future researches. Clinical features of 2019 novel coronavirus pneumonia presented gastrointestinal symptoms but without fever onset. arxiv Computeraided diagnosis with deep learning architecture: applications to breast lesions in us images and pulmonary nodules in ct scans Xception: Deep learning with depthwise separable convolutions Covid-19 image data collection Imagenet: A large-scale hierarchical image database Rapid ai development cycle for the coronavirus (covid-19) pandemic: Initial results for automated detection & patient monitoring using deep learning ct image analysis Identity mappings in deep residual networks Review of the clinical characteristics of coronavirus disease 2019 (covid-19) Optimal deep learning model for classification of lung cancer on ct images Artificial intelligence distinguishes covid-19 from community acquired pneumonia on chest ct A survey on deep learning in medical image analysis Coronavirus disease 2019 (COVID-19): Epidemiology, virology, clinical features, diagnosis, and prevention COVID-19 CORONAVIRUS PANDEMIC Automatic detection of coronavirus disease (covid-19) using x-ray images and deep convolutional neural networks Sperm detection and tracking in phase-contrast microscopy image sequences using deep learning and modified csr-dcf U-net: Convolutional networks for biomedical image segmentation Lung infection quantification of covid-19 in ct images with deep learning Emerging 2019 novel coronavirus (2019-ncov) pneumonia. Radiology Clinical features of severe pediatric patients with coronavirus disease 2019 in wuhan: a single center's observational study Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest radiography images A deep learning algorithm using ct images to screen for corona virus disease Deep learning system to screen coronavirus disease 2019 pneumonia Chest ct severity score: An imaging tool for assessing severe covid-19 Deep learning analysis of the myocardium in coronary ct angiography for identification of patients with functionally significant coronary artery stenosis We would like to appreciate Joseph Paul Cohen and the others who provided these x-ray images from patients infected to COVID-19. We thank Linda Wang and Alexander Wong for making their code available, in which we have used some of it on our research for preparing our dataset. We also thank Google Colab server for providing free GPU.