key: cord-1049267-itnfh8fk
authors: Bohmrah, Maneet Kaur; Kaur, Harjot
title: Classification of Covid-19 patients using efficient Fine-tuned Deep learning DenseNet Model
date: 2021-08-13
journal: Global Transitions Proceedings
DOI: 10.1016/j.gltp.2021.08.003
sha: 6ca3bd8625b792a643c1fb06a63bc4aa6bfbfd89
doc_id: 1049267
cord_uid: itnfh8fk

As COVID-19 pandemic caused completely spoils the livings, almost more than one year passed, still lives were not on the track. It is important to diagnose the COVID-19 patients earlier and provide the prompt treatment. The Convolutional Neural Network (CNN), a deep neural network that specializes in image processing and image classification. In this paper, a fine tuned DenseNet201 model was proposed which is used to classify Chest X ray images. Firstly, different DenseNet121, DenseNet169 and DenseNet201 model trained and tested on the same dataset. With the experiment, it is observed that DenseNet201 model performs well as compared to other dense models. Furthermore, DenseNet201 experiments over different optimizers and it is noticed that RMSprop, Adagrad and Adamax performs better. Proposed model achieves accuracy of 95.2% as compared to other models. We experimentally determine that RMSprop optimizer with DenseNet201 produces better results as similar to Adam and Adamax widely used optimizers.

In an unforeseen situation, our civilization is yet again fighting a novel coronavirus, SARS-CoV-2 (Severe Acute Respiratory Syndrome Coronavirus-2), in another historical battle. According to a study, the virus originated in Wuhan, China, around December 2019. As Soon after, the coronavirus was dubbed COVID-19, and due to it communicable properties spreads quickly across the world, prompting the World Health Organization (WHO) to proclaim a global pandemic in March 2020 [1] .

However, given the significance of the human community, it has become necessary to develop an autonomous approach that can diagnose COVID-19 in a short amount of time. A crucial key initiative is accurate and effective COVID-19 patient detection, so that positive cases undergo prompt treatment and are properly segregated from the general population; a precaution believed critical in preventing the spread of the disease [2, 3] . In recent years, AI techniques have been widely used in a variety of challenges ranging from classification, segmentation, and face identification to upgrade-resolution and image enrichment in computer vision and medical image analysis [4] . As COVID-19 is a respiratory disease which directly harms the lungs and cause difficulty in breathing. For the diagnosis of COVID patient's chest X-ray images plays a vital role. For the accurate and fast results these images can be examined using the Deep learning techniques which is already proves its excellence in the field of image processing and analysis [5, 6] .

In this paper, a novel approach is presented for the detection of COVID-19 images using deep learning DenseNet201 model. The suggested DenseNet model modification ensures flow of information by directly connecting all layers in the network to their feature maps. The feedforward method is maintained by obtaining additional values from prior layers and passing feature maps from the previous layers on to all subsequent levels [7] . Using the TensorFlow Python library, the suggested model was tested in practice and yielded encouraging results for analysis of the data. The main contribution of this can be summarized as follows:

 Provide a literature survey, authors using DenseNet model for the classification of X-ray images.

As none of paper explains the different DenseNet models and claims which optimizer provide the best results.  We conduct an extensive evaluation on three different DenseNet models using same dataset and the result shows DenseNet201 performs well which further used to design a fined tuned DenseNet model, a proposed model.  DenseNet201 shows the best result which is further used to design proposed model. Pre-trained DenseNet model tuned and trained over the different optimizers.  RMSprop and Adagrad, Adamax optimizers perform well with the DenseNet201 model.

The rest of the paper is structured as follows: section 2 represents the related work; section 3 defines the concepts of DenseNet model with different number of layers. Further, section 4 describes the different optimizers used in the article and section 5 represents the proposed work followed by result analysis done in section 6. Last but not the least conclusion is summarized in section 7.

Computer vision assists us in developing autonomous systems that do tasks that are similar to and, in some circumstances, better than human vision. One of the most important contributions of computer vision is in the field of medicine improved illness diagnosis, therapy, and prevention using data on medical imaging [8] . A deep neural network excellently performs in the field of image processing and Convolutional Neural Networks (CNN) widely used for this purpose. As there are various forms of CNN such as Resnet, VGG, Xception and many more [9] . This section represents the deep learning techniques used for the classification of COVID chest X-ray images. Moreover, abundant of research work done on CNN model used for COVID-19 detection but this paper concentrate only on the DenseNet CNN model, hence we explored only those author's work proposed by various researchers and experts where DenseNet model used in their research works.

In this section, we focus on deep learning DenseNet model which also a type of CNN model. Further, we summarize some related and existing work where DenseNet model used to classify the COVID-19 chest X-ray images. In the paper [10] , author experiments the various deep learning models and DenseNet121, DenseNet169 and DenseNet201. Total 746 images were used by the author for the training and testing of the models. With the experiments, author shows DenseNet201 performs excellently as compared to other deep learning models. In the paper [11] , author proposed a CNN model and compared with pretrained deep learning model. Author used two data sets of 111 images and 6290 chest X-ray images. Author also used DenseNet201 for the comparisons. Though in his experiments, DenseNet201 doesn't perform well enough.

In the article [12] , author proposed a fused DenseNet model which is a concatenation of two DenseNet model. Furthermore, author used fused DenseNet for the classification of 9208 chest X-ray images and achieved accuracy of 97.99%. In the article [13] , author runs various deep learning models for the classification of covid19 patients. Author also includes DenseNet121 model for the experiment and total 5000 images were used to train and test the model.

In the article [14] , author proposed a model by fine tuning the pretrained DenseNet201 model and further used for the classification of 950 chest X-ray images. In the paper [15] , author used DenseNet CNN model to classify the 2482 CT scan images. Preprocessed and augmented dataset used for training of the model.

In the paper [16] , author proposed DenseNet-OTLS for the effective detection of COVID-19. All the three DenseNet121, DenseNet169 and DenseNet201 were used for the experiment and total 640 images were used to train and test the model. Author worked on two factor single learning rate and Composite learning rate for the optimization of DenseNet model.

In the article [17] , author proposed a CovidDenseNet model using DenseNet121 for the diagnosis of COVID-19. Author experiments both two and three class classification and patient's wise k-fold was also performed more than 13800 chest radiography images were used for the experimentation. Further, Grad-CAM used to highlight the infected regions of lungs for better clarity in reading images.

With the study of all the articles mentioned above DenseNet CNN model performs well as compare to other CNN models but there is a gap which DenseNet model performs well under which circumstances and furthermore we try to explore the working of different optimizers with this research article.

Deep learning [18] focuses on several levels of abstraction, with higher layers representing more abstract data knowledge. As an alternative to traditional machine learning techniques for identifying COVID-19 chest X-ray images, neural networks can be used. Every layer of the CNN in DenseNet is interconnected to every other layer in the network in a feed-forward way, which reduces the chance of gradient vanishing, reduces the number of parameters to train, reuses feature maps, and each layer takes all previous layer features as inputs [19, 20] . Table  1 explains the dense layers for the different DenseNet models. Mainly, DenseNet model consist of many dense blocks and one dense block contains convolutional layer, ReLU layer and batch normalization. On the other hand, two dense blocks connected with convolutional and max pooling layer and the last dense block connected with global average pooling and softmax classifier [21] . 

Optimization Algorithms, also known as Optimizers, are essential for improving the accuracy of a neural network. In a traditional approach, hyper parameters of a model were tuned using optimizers according to their design. An optimizer's job is to manipulate the weights and learning rates of our model's nodes during the training phase so that the loss function is effectively minimized [22, 23] .

Stochastic Gradient Descent or SGD is a version of Gradient (or slope of a function) Descent, which is the simplest basic optimization algorithm. Although too simple to be employed in Deep Learning, the latter has a wide range of applications in Linear Regression, Classification, Backpropagation, and other areas, with the benefit of ease of computation and implementation. Stochastic (means random) Gradient Descent, on the other hand, chooses a few sample data rather than the complete dataset for each round, resulting in a significant increase in estimating speed [24] [25] [26] .

RMSprop is a gradient-based optimizer that uses an adaptive learning rate that changes over time rather than considering the learning rate as a hyper-parameter. RMSprop, or Root Mean Square Propagation, has an unusual fact related with it: despite its popularity, it is an optimizer that has never been released. In his online course on Neural Networks for Machine Learning, Geoff Hinton, the father of backpropagation, proposed it.

Although the Adaptive Gradient Technique (Adagrad) is quite similar to the stochastic gradient descent algorithm, show that it does not use adaptive gradients to improve robustness. One of Adagrad's key advantages is that it eliminates the need for manual learning rate calibration, whereas it's most serious fault is the aggregation of squared gradients in the denominator.

AdaDelta is a modified successor to Adagrad and another part of the family of stochastic gradient descent algorithms. The default learning rate has been removed from the update rule, thus there is no need to change its default value. It also offers adaptive hyper-parameter tweaking approaches and is visibly resistant to chaotic gradient details. By providing a fixed size, it imposes constraints on the aggregate previous gradients [27] [28] [29] .

The most prevalent optimizer is Adaptive Moment Estimation, or Adam. It has a lot in common with RMSprop and Adagrad. Adam employs the L2 norm, often known as the Euclidean norm, for optimization. It is known for its great efficiency, adaptability, and speedier convergence [30] .

Adamax and Adam were both introduced in the same paper. It's a variation of the Adam optimizer that uses infinity or max norm for optimization. Adamax will outperform Adam with data that is conventionally unstable in terms of gradient updates (for example, a dataset with several misfits) [31, 32] .

In this article, author proposed a new model using the pre-trained DenseNet201 model. This study chose to employ a densely connected neural network, the DenseNet model, for the proposed model, as shown in Fig. 2 . The goal of using a DenseNet model is to ensure that features propagate flawlessly across the network without efficiency depletion, even as the depth is increased. Let summarized the whole process of proposed model in further paragraphs.

The proposed model evaluated on Covid data set collected from Kaggle repository. The data set consist of chest X-ray images categorized into 3 classes Covid, normal and viral pneumonia and further this data set divided into train and test folders for the experimentation work. The train data set containing 111 images of Covid,70 images of normal and 70 images of viral pneumonia. For the validation 20% of data from train folder taken. The test data set contains 26 images of Covid, and 20 images of normal and 20 images of viral pneumonia. Figure 3 explains the sample images from the data set it is clearly observed the normal image of chest X-ray contains the white opacity surrounded by the lungs were as this opacity reduces in case of viral pneumonia and this opacity almost zero for Covid chest X-ray images.

The main principle behind transfer learning is that the skills learned when studying a model can be applied to a different learning assignment. CNNs are based on a series of deeper layers, with input passing through a number of them. The input data may be lost before it reaches the network's final layer.

Each layer in DenseNet receives extra inputs from all preceding levels and sends its own feature-maps to all subsequent layers. Each layer receives a collective knowledge from the levels above it. Because each layer receives feature maps from all previous layers, the network can be thinner and more compact, resulting in fewer channels (and hence improved computing efficiency). Fig. 1 explains the architecture of DenseNet model used for image classification. 

Hyper-parameters are the parameters which plays an important role in training the model and values selected for the hyper-parameters largely affects the model accuracy. Most commonly known hyper-parameters for the deep learning model are learning rate, batch-size, number of epochs, number of dense layer and weights. The hyper-parameter defines the proposed model were mentioned in the table 2. The overall design of the proposed fine-tuned DenseNet201 model defined in the Fig. 2 . DenseNet201 model is used for the proposed methodology as we already performed experiment with DenseNet121, DenseNet169 and DenseNet201 on the same data set. For the proposed methodology we used the concept of data augmentation on the chest X-ray images. After data augmentation images were ready for the input to the model and different optimisers SGD, RMSprop, adagrad, adadelta, adam and adamax with the same learning rate. All the top layers containing the dense blocks were freeze as these layers already trained on the ImageNet database. Further, dense layer with the activation function at output layer softmax is added for the classification of data set containing three classes normal, Covid and viral pneumonia. We trained are model by using the hyper-parameters already defined in table 2. Complete model runs for 10 epochs with the batch size equal to 3. 

To evaluate categorization predictions, four types of metrics were computed.

label prediction done by model for an input image and image actually belongs to the same label i.e. an Chest X-ray that is classified as COVID and is actually belongs to COVID Class.

label prediction made by model for an input image and image actually does not belong to that class i.e. model predicted an image does not belong to NORMAL class and that image actually not NORMAL. 

In this section, we discuss the various computed results. The first part of our experimentation is the exploration of different DenseNet models. Table 3 explains the accuracy and other evaluation metrics for the entire three different layers DenseNet model. Further, we experiments the different optimizers and the figure 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 and fig. 15 shows the training and validation accuracy and loss against the different optimizers. Table 4 represents the performance of all the optimizers used in the experimentation work whereas fig. 10 and fig. 11 shows the performance of the proposed methodology. On the other hand, with the help of confusion matrix one can understand the how accurately a model identify the images. The diagonals values should be higher as compared to the other values for a model represents the higher performance. Fig. 16 to fig. 20 represents the confusion matrix and fig. 18 is the confusion matrix for the proposed methodology. Table 5 compares the proposed methodology with the existing art of work. 

In this paper, a fine tuned DenseNet model proposed for the classification of COVID-19 patients. DenseNet121, DenseNet169 and DenseNet201 were experimented on the same dataset and with the same hyper-parameters values and RMSprop optimizer. DenseNet201 provides better results as compared to the other DenseNet models. DenseNet201 tuned and using RMSprop optimizers performs well. For future work, more research can be done on the DenseNet201 and optimization of these models for accurate and effective classification of COVID-19 using Deep learning models. While the results are intriguing, more research on a larger sample of COVID-19 photos is needed to provide a more credible estimate of accuracy rates.

The continuing 2019-nCoV epidemic threat of novel coronaviruses to global health-the latest 2019 novel coronavirus outbreak in Wuhan

Rapid ai development cycle for the coronavirus (covid-19) pandemic: Initial results for automated detection & patient monitoring using deep learning ct image analysis

Radiologist-level pneumonia detection on chest x-rays with deep learning

Artificial intelligence and COVID-19: deep learning approaches for diagnosis and treatment

Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks

Determination of disease severity in COVID-19 patients using deep learning in chest X-ray images. Diagnostic and interventional radiology

Deep neural networks for object detection

IEEE Conference on Computer Vision and Pattern Recognition

Review of deep learning algorithms and architectures

An approach for recognizing COVID-19 cases using Convolutional Neural Networks applied to CT scan images

Problems of deploying cnn transfer learning to detect covid-19 from chest x-rays

Diagnosing Covid-19 chest x-rays with a lightweight truncated DenseNet with partial layer freezing and feature fusion

Deep-covid: Predicting covid-19 from chest x-ray images using deep transfer learning. Medical image analysis

Optimization and Fine-Tuning of DenseNet model for classification of Covid-19 cases in Medical Imaging

DenseNet Convolutional Neural Networks Application for Predicting COVID-19 Using CT Image

Covid-19 diagnosis via DenseNet and optimization of transfer learning setting

Covid-DenseNet: A deep learning architecture to detect covid-19 from chest radiology images

Deep learning

Machine learning-based prediction of COVID-19 diagnosis based on symptoms. npj digital medicine

Deep learning for visual understanding: A review

An experimental approach towards the performance assessment of various optimizers on convolutional neural network

Deep learning for visual understanding: A review

Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms

Fingerprint enhancement based on tensor of wavelet subbands for classification

Offline Signature Recognition Using Image Processing Techniques and Back Propagation Neuron Network System

Adam: A method for stochastic optimization

Uplink and downlink NOMA transmission using full-duplex UAV

Demand side management of small scale loads in a smart grid using glow-worm swarm optimization technique

Adadelta: an adaptive learning rate method

MyWebGuard: toward a user-oriented tool for security and privacy protection on the web

Analysis and computation of encryption technique to enhance security of medical images

Towards Real-time and Efficient Cardiovascular Monitoring for COVID-19 Patients by 5G-Enabled Wearable Medical Devices: A Deep Learning Approach‖, Neural Computing and Applications