key: cord-0706836-5thq2m3v
authors: Kundu, Rohit; Singh, Pawan Kumar; Mirjalili, Seyedali; Sarkar, Ram
title: COVID-19 Detection from Lung CT-Scans using a Fuzzy Integral-based CNN Ensemble
date: 2021-10-01
journal: Comput Biol Med
DOI: 10.1016/j.compbiomed.2021.104895
sha: 41203302888fab0fbfa4174f3b5512a3b9779ca2
doc_id: 706836
cord_uid: 5thq2m3v

The COVID-19 pandemic has collapsed the public healthcare systems, along with severely damaging the economy of the world. The SARS-CoV-2 virus also known as the coronavirus, led to community spread, causing the death of more than a million people worldwide. The primary reason for the uncontrolled spread of the virus is the lack of provision for population-wise screening. The apparatus for RT-PCR based COVID-19 detection is scarce and the testing process takes 6-9 hours. The test is also not satisfactorily sensitive (71% sensitive only). Hence, Computer-Aided Detection techniques based on deep learning methods can be used in such a scenario using other modalities like chest CT-scan images for more accurate and sensitive screening. In this paper, we propose a method that uses a Sugeno fuzzy integral ensemble of four pre-trained deep learning models, namely, VGG-11, GoogLeNet, SqueezeNet v1.1 and Wide ResNet-50-2, for classification of chest CT-scan images into COVID and Non-COVID categories. The proposed framework has been tested on a publicly available dataset for evaluation and it achieves 98.93% accuracy and 98.93% sensitivity on the same. The model outperforms state-of-the-art methods on the same dataset and proves to be a reliable COVID-19 detector. The relevant source codes for the proposed approach can be found at: https://github.com/Rohit-Kundu/Fuzzy-Integral-Covid-Detection.

COVID-19 is an extremely contagious disease that led to a community spread in the world. It was declared to be a public health emergency by the World Health Organization (WHO) in January 2020 and recognized as a pandemic in March 2020. Currently, more than 50 million people worldwide have been affected by the coronavirus, out of which it accounts for more than 1.25 million unfortunate deaths (2.5% mortality rate). However, many positive cases go undetected due to the lack of provision for conducting population-wise testing. The current statistics of the COVID-19 pandemic in the world are shown in Figure 1 . All the data for the graphs have been obtained from a publicly available source by [47] .

The available standard tests for COVID-19 detection is the Real-Time Polymerase Chain Reaction (RT-PCR) and the rapid antigen test. However, there are some drawbacks of the RT-PCR test [59] . First, it takes 6-9 hours for the testing process. Secondly, there is the issue of unavailability of sufficient apparatus for population-wise testing, and thirdly, the RT-PCR test is not wholly reliable, being only 71% sensitive [67] . Due to these reasons, this virus has spread to an uncontrollable extent. On the other hand, the rapid antigen test [34] uses the human blood sample to detect the presence of IgG and IgM antibodies for COVID-19 detection and can produce the result within 15 minutes. However, the human body takes several days to produce these antibodies after the virus has entered the host, making the antibody test unreliable to detect COVID-19 in its early stages. This can lead to the spread of the virus even before the disease is diagnosed. Even though several vaccines have been developed by experts across the world, it will take a lot of time to vaccinate each individual on the planet, especially with the spread of the new variants of the COVID-19 virus that are emerging due to gene mutation in the virus. Therefore, there is a need to develop Computer-Aided Detection tools for faster and less laborious screening procedures to help control the pandemic. A large part of this research focuses on using deep learning techniques [42, 9] . Deep learning is a machine learning technique that uses artificial neural networks for classifying structured or unstructured data through a complex decision-making process.

As compared to the RT-PCR test, chest CT-scans are more sensitive [14, 1] , and more widely available. Although X-Ray images are being used for automated screening, CT-scan images give a more detailed view of the lungs and therefore, are more reliable for this study [39] . The most distinguishing feature of COVID-19 infected chest CT scans is the "ground-glass opacities" (GGO) that appear inside the lungs [33, 11] . They emerge when the alveoli (tiny air sacs) get filled with fluid and appear as a white frosted-glass like appearance in the chest CT- scan [43] . Figure 2 shows the chest CT-image of a COVID-19 infected patient and a COVID-negative patient. The GGO in the lungs of the infected patient are marked in the image.

Deep learning has been proven successful in biomedical supervised learning applications like [15, 2] . In the current image classification task, transfer learning has been employed which refers to the method of re-using a deep learning model used for a specific task, on a separate but related task. It is used when the current problem lacks sufficient data for training the model from scratch. Here, the parameters from the previous task are loaded, and the data from the current task are used for finetuning the model. The present work uses the Ensemble Learning technique for classifier fusion. In doing so, four pre-trained models, namely, VGG-11 by Simonyan et al. [53] , GoogLeNet by Szegedy et al. [57] , SqueezeNet v1.1 by Iandola et al. [24] and Wide ResNet-50-2 by Zagoruyko et al. [69] have been used, and Sugeno Fuzzy Integral has been used for ensembling the four classifiers and generating the final prediction scores for the images. Ensemble Learning enhances the performance of the constituent models by accounting for increased diversity in predictions. Although simple fusion schemes, such as averaging probability scores, majority voting, etc., have been used in literature, such techniques cannot account for the complex co-occurrences within the data to be classified. They do not account for the different weights of classifiers based on the decision scores obtained at testing time. Therefore, there is a need to generalize [54] used in this study. the simple fusion schemes, such that the weightage of one classifier may be conditioned upon the weightage of the others, to promote the adaptive importance of different classifiers for every sample image.

In the present work, the Sugeno fuzzy integral has been used for ensembling the aforementioned classifiers to address the shortcomings of using simple fusion techniques. Fuzzy integrals [61] are effective aggregators, that use the degree of uncertainty in the decision scores as additional information for the fusion of classifiers. It can be viewed as a generalisation of aggregation operators on a set of confidence scores that use some weightage given to each information source, called fuzzy measures. The Sugeno fuzzy integral takes into consideration the confidence of predictions by the base learners to assign adaptive weights to them for each input to the model, unlike the simple fusion schemes found in the literature that use fixed predetermined weights. This concept and its application in the present study are discussed in detail in Section 3.5.

The rest of the paper has been organized into sections as follows: Section 2 provides a brief literature survey of existing COVID-19 detection models; Section 3 explores in detail, the methodology proposed in the current study; Section 4 describes the results obtained on implementation of the proposed model on a publicly available dataset and finally, Section 5 concludes the findings from this paper and outlines some possible extension of this work.

Since the outbreak of the COVID-19 pandemic, a diverse range of classification models have been proposed for the automated detection of COVID-19 patients [28] . A large number of these methods use chest radiology images (X-Rays) like [5, 63] . Mahmud et al. [38] developed the CovXNet model for differentiating COVID-19 cases from pneumonia using chest X-ray images, wherein they achieved 90.2% accuracy on the multiclass classification problem. Ozturk et al. [42] developed the DarkNet model for the binary and multi-class classification of COVID-19 and pneumonia using chest X-Ray images. They based their model on the You Only Look Once (YOLO) architecture imbibing some modifications and achieved an accuracy of 98.08% for the binary classification problem (COVID-19 vs. No Findings) and 87.02% for the multi-class problem (COVID-19 vs. Pneumonia vs. No Findings).

However, chest CT-scan images are more sensitive as they give a clearer view of the lungs, therefore enabling a more reliable diagnosis. Shi et al. [51] segmented the lungs from the CT-scan image and used machine learning techniques for location-specific feature extraction. The extracted features were fed to their proposed "infection Size Aware Random Forest (iSAR)" classifier for the final classification. Incorporating 5fold cross-validation on their dataset, they obtained 87.9% accuracy, 90.7% sensitivity and 83.3% specificity.

Machine learning requires the extraction of handcrafted features and often results in redundant features being extracted. Deep learning, on the other hand, learns the important features on its own and provides an end-to-end classification system, and therefore has been used for developing several COVID-19 detection models like [32, 50] . Gozes et al. [16] used a fusion of two deep learning models for the classification of COVIDinfected patients. One model uses 2D slices of CT scans and the other uses volumetric CT-scan images. Xu et al. [66] proposed a location-attention classification model that uses features extracted by the ResNet-18 pre-trained model from the lung CT images for classification. Burdick et al. [10] conducted a multicentre clinical trial to predict invasive mechanical ventilation of COVID-19 infected patients within 24 hours of the initial encounter with the virus. However, acquiring data from clinical trials requires authorization from several bodies and is not publicly accessible.

Hu et al. [22] developed a weakly supervised COVID classification model that first performs lung segmentation and then uses a deep CNN model that extracts features at 3 different steps for multi-scale learning. Ni et al. [41] proposed a deep learning algorithm for lesion detection, segmentation and segmentation to find pneumonia lesions in COVID-19 patients.

An elaborate study on the efficacy of different transfer learning models (popular and new models) for COVID-19 detection using chest CT images have been conducted by Ardakani et al. [6] . They found ResNet-101 to be the most accurate classifier, obtaining an AUC of 0.994 on a dataset of 1082 CT images. Amyar et al. [3] developed an encoder-decoder network for the segmentation and classification of chest CT images for COVID-19 detection. For this, they use an encoder coupled with two decoders and a multi-layer perceptron for the segmentation, reconstruction and classification tasks. They achieved an average AUC of 97% on a dataset of 1349 patients' lung CT scans.

Carvalho et al. [12] developed a framework for COVID-19 classification using CT images wherein they extracted deep features from a CNN model and then used a Genetic Algorithm for optimal feature subset selection and classification. Vinod et al. [62] proposed a "Deep Covix-Net" model for detecting COVID-19 from both X-ray and CT images. They tested their model in both binary-class and multi-class settings achieving accuracies of 97% and 96.8% respectively.

Most of the existing methods use a single classification model. The ensemble learning paradigm is very seldom explored for addressing the COVID-19 detection problem. Thus in this paper, we develop a deep ensemble model for the classification of chest CT images for COVID-19 detection. However, instead of using popular ensemble approaches prevalent in literature, like probability averaging, or weighted probability averaging, we use a fuzzy integral-based ensemble. This leverages adaptive priority to the classifiers on the fly instead of the classical ensemble methods that set the priority given to each classifier beforehand.

The COVID-19 pandemic has caused the front-line workers to work tirelessly for attending to COVID-infected patients, along with patients homing other diseases, putting their lives at risk. Also, there is no sign of the pandemic stopping, and since population-wise screening is impossible with the current level of RT-PCR test kits available. So, the need for the development of an automated screening model motivated us to conduct this study.

In this paper, we propose a method to improve the efficiency of the Convolutional Neural Network (CNN) models by using a fuzzy integral based ensemble. Four distinct CNN architectures are used to ensure that there are multiple sources of information, manifested by the individual characteristics of the different features. The overall workflow of the proposed model is shown in Figure 3 . The complementary nature of the data captured by the different CNNs is verified by comparing the statistical divergences from the decision scores of each CNN model. The CNN classifiers are fused by applying the Sugeno Fuzzy integral using which has two advantages, first, it works as a generalization of classical empirical schemes, and second, conditioning the weights of each model based on the probability scores of the individual classifiers capture the complementary information more robustly.

The main contributions of this paper are as follows:

1. Chest CT-scan images are used to develop the proposed model in this study which is both widely available and can be performed faster compared to the tedious RT-PCR testing process. 2. To address the challenge of low availability of publicly available chest CT data of COVID-19, transfer learning models have been used in the first phase of the framework. The complementary natures of the features vectors obtained from the CNNs are verified using the Kullback-Leibler Divergence and Jensen-Shannon Divergence. 3. For using the features of the various models, an ensem-ble technique has been used based on a fuzzy measure: the Sugeno Integral. Fuzzy integrals are a powerful fusion technique for using the characteristics of all the input information sources. 4. The proposed method outperforms the existing state-ofthe-art models achieving very high accuracy (98.93%) and sensitivity (98.93%) compared to the 71% sensitivity of the RT-PCR test.

The proposed framework uses four pre-trained deep learning models, which are trained on the ImageNet dataset [13] . The ImageNet dataset is a huge data source consisting of 14 million images belonging to 1000 classes. Training the models on such a large variety of images set the model parameters (weights and biases of the nodes of each layer) optimally, and only minor fine-tuning is generally required for using the trained models on a different classification task. The deep transfer learning models used in this study are:

1. VGG-11 by Simonyan et al. [53] 2. GoogLeNet by Szegedy et al. [57] 3. SqueezeNet v1.1 by Iandola et al. [24] 4. Wide ResNet-50-2 by Zagoruyko et al. [69] VGG-11 is a very deep network whereas GoogLeNet has inception modules that contain parallel convolutions and hence, both of these networks have a high number of parameters. On the other hand, both SqueezeNet v1.1 and Wide ResNet-50-2 5 J o u r n a l P r e -p r o o f Table 1 . These pre-trained models are used for the binary classification task at hand and the prediction probability scores for the images are stored. These probability prediction scores are then used for computing the Sugeno Fuzzy Integral to calculate the final predictions on the test set and then the predictions are used to compute the evaluation metrics. The transfer learning models and the fuzzy integral ensemble method adopted in this study are described in the following subsections.

The VGG models developed by the Visual Geometry Group [53] , are some of the deepest CNNs in literature. The introduction of several weight layers (up to 16-19 layers) was made possible by decreasing the size of the convolution filters to 3x3 kernels. The VGG group stresses the fact that the depth of a CNN model is important for visual representations and beneficial for accurately applying to several genres of classification tasks in Computer Vision. The VGG-11 CNN architecture has been shown in Figure 4 .

The GoogLeNet CNN model by Szegedy et al. [57] , the name of which is inspired from LeNet-5 [31] is 22 layers deep network that consists of "inception modules" instead of uniformly increasing networks. The inception block inspired by Lin et al. [35] , allows a large number of units to be accommodated at each stage due to the parallel convolution and pooling, without facing an uncontrolled computational complexity because of the increased number of parameters. For controlling the computational complexity, the architecture uses dimensionality reduction in the inception blocks as shown in Figure 5(b) , rather than the naive inception block ( Figure 5(a) ).

The GoogLeNet model proves that an optimal sparse architecture made from the available dense building blocks, i.e., the introduction of inception blocks, improves the performance of neural networks for Computer Vision tasks. The architecture of the GoogLeNet CNN model has been shown in Figure 6 .

The SqueezeNet [24] is a CNN model having 50 times lesser parameters than AlexNet [26] model but achieving similar accuracies on the ImageNet data classification task. However, to employ model compression techniques [19, 18] allow the SqueezeNet model to be stored under 0.5MB space which is 510 times smaller than the space occupied by the AlexNet model.

The salient features of the SqueezeNet architecture are as follows:

1. Majority of 3x3 filters have been replaced with 1x1 filters, which have 9 times fewer number parameters. 2. To maintain a small total number of parameters, the number of input channels has been decreased to only 3x3 filters using "squeeze layers". 3. The down-sampling of the images has been performed late in the architectures for obtaining larger activation maps from the convolution layers. 4. A "fire module" has been introduced, which is comprised of a "squeeze" convolution layer that feeds into an "expand" layer that is a mix of 1x1 and 3x3 convolutions. The fire module is shown in Figure 7 . The squeeze layer in the fire module helps to limit the number of input channels.

The following hyperparameters describe a fire module: s 1 denotes the number of 1x1 filters in the squeeze layer; e 1 denotes the number of 1x1 filters in the expand layer, and e 3 denotes the number of 3x3 filters in the expand layer.

The architecture of SqueezeNet v1.1 is shown in Figure 8 .

Wide ResNets [69] are a genre of CNNs honing residual blocks in their architecture. They have lower depths and greater width than the traditional ResNets [20] achieving state-of-theart results but consuming much lesser training time. They have proven that the 50 layers deep Wide ResNet (Wide ResNet-50-2) outperformed the 152 layers deep ResNet (ResNet-152) on the ImageNet [13] data classification task. Similar to the previous networks, the final classification layer of the Wide ResNet-50-2 architecture was changed for the binary classification task in the present study. The various residual blocks used in the Wide ResNet architecture has been shown in Figure 9 , and the complete architecture has been shown in Figure 10 

Ensemble learning is a strategy for fusing the salient properties of two or more base learners. Such a framework performs more robustly than its constituent models because ensembling reduces the variance in the prediction errors. However, most of the traditional ensemble frameworks [30, 29] tend to assign a pre-defined classifier weight to compute the ensemble. In this study, we use a fuzzy integral-based approach, wherein instead of assigning a fixed weight to the classifiers, we leverage adaptive importance on the fly while testing. Such a method takes into account the confidence in the predictions of the base learners for every sample to assign importance to the classifier, and thus performs more robustly than the traditional approaches in the literature.

Fuzzy integrals have proven successful in several pattern recognition problems like [65, 36] . They are powerful and flexible functions for aggregating information based on a fuzzy measure. This fuzzy measure represents the relevance or importance of the constituent information sources when the aggregation is computed.

A fuzzy measure [55] is the set function f that satisfies the following properties:

where β is a Borel field [37] of an arbitrary set X. The Sugeno fuzzy-λ measures was introduced by Tahani et al. [60] . Suppose there is a set of scores S = {s 1 , s 2 , s 3 , . . . , s N } where N is the number of information sources (N = 4 in our case), and e ∈ S . The Sugeno-λ measure is the function f λ : 2 S → [0, 1] such that it satisfies the following conditions:

Therefore, λ is the real root of the equation as given in Equation 2.

The Sugeno Integral [56] is a fuzzy measure integral [46] that can be defined as follows: Suppose (A, µ) is a measurable (Borel) space, and f : X → [0, 1] is a µ-measurable function, then the Sugeno Integral of the measurable function f with respect to a fuzzy measure Ω is given as in Equation 3 .

where 

This section describes the dataset used in the current study, and the results obtained by implementing the proposed approach on the dataset. Comparisons to existing standard models in literature have also been drawn to validate the efficacy of the framework. 

Category Total Train set Test set  1  COVID  1252  876  376  2 Non-COVID 1229 860 369

The dataset used in the present work is proposed by Soares et al. [54] which they generously made publicly available on Kaggle 1 . It consists of a total of 2481 chest CT-scan images divided into two categories: COVID and Non-COVID, distributed unevenly among the two classes. The distribution of images in the training and test set from each class are shown in Table 2 . The images in the dataset have been classified by Soares et al. [54] according to the results from the RT-PCR tests conducted on the patients. Patients with confirmed COVID or Non-COVID cases are only present in the dataset and thus, there is little or no risk of unnecessary bias through the non-representative selection of control patients.

The current image classification problem is a binary classification task and the metrics that have been used for evaluating the performance of the proposed approach on this task are accuracy, specificity, precision, recall or sensitivity and f1-score. The definitions for "True Positive (TP)", "False Positive (FP)", "True Negative (TN)" and "False Negative (FN)" are required to understand and formulate these parameters.

The two classes in the binary classification task are the COVID-19 positive class and the COVID negative class. If the lung CT-image is of a COVID-19 positive patient and the model classifies it to be COVID positive class (i.e., correct classification), then the sample image is said to be True Positive (TP). Now, if the lung CT image is of a COVID positive patient, but the model classifies it to be COVID negative, the sample is called False Positive (FP). Similarly, a COVID negative lung CT image when classified correctly is called True Negative (FN). And, a COVID negative CT-scan image misclassified as COVID positive is called False Negative (FN). The evaluation metrics can now be formulated as Equations 4, 5, 6, 7 and 8.

Recall or S ensitivity = T P T P + FN (7) In the COVID-19 classification task, the number of FP cases is more dangerous, because COVID-19 is an extremely contagious disease, and a COVID-19 positive patient whose diagnosis is determined as COVID negative will infect other people as a super-spreader. However, the FP case is not that big a problem, since a COVID negative patient diagnosed as a positive needs to just maintain precautions under prescribed guidelines since it is a disease with a very low mortality rate. The contagiousness of a virus is inversely related to its deadliness.

The deep transfer learning models used in the current study have been trained for 100 epochs on the COVID-19 binary classification dataset of lung CT-scan images. The hyperparameters used for the model training are shown in Table 3 which have been set experimentally. The loss curves obtained on training by the four models are shown in Figure 11 .

The probability distribution from the four classifiers has been fused using the Sugeno Fuzzy Integral as described in Section 3.5. The confusion matrix of the final predictions on the test set is shown in Figure 12 .

The class-wise results obtained by the proposed framework on the test set of the SARS-COV-2 dataset are shown in Table 4 .

The most commonly used tests for COVID-19 detection are the RT-PCR test and the rapid antigen-antibody test. The RT-PCR test takes a long time to produce results and is not sufficiently sensitive. The rapid antigen-antibody test relies on the detection of formed antibodies in the human body and produces results in 15 minutes. However, the human body requires several days to form antibodies, and thus in the initial stage, an infected patient may go undetected for days. The proposed automated framework gives a high classification accuracy and sensitivity by using chest CT-scan images while also being a much faster method. The proposed method can be used as a plug-andplay model, where new test images can be passed through the model to get the predictions by the ensemble mechanism. Thus, the incorporation of the proposed method in the field under consideration is simplistic and COVID-19 cases can be screened more efficiently.

For establishing the complementary nature (dissimilarity) in the probability vectors generated by the four pre-trained CNN models, two statistical divergence methods are incorporated: J o u r n a l P r e -p r o o f Table 6 : Comparison of the proposed framework with some standard pretrained models on the SARS-COV-2 dataset [54] .

Accuracy DenseNet161 [23] 96.91% Wide ResNet-50-2 [69] 96.78% VGG-11 [53] 96.38% SqueezeNet v1.1 [24] 96.24% GoogLeNet [57] 96.11% Inception v3 [58] 92.15% Proposed Method 98.93%

the Kullback-Leibler Divergence (KLD) [27] and the Jensen-Shannon Divergence (JSD) [40] . KLD, is closely related to relative entropy, is a measure of dissimilarity between two probability distributions, and is an non-symmetric measure. Suppose we have a probability space X. For every discrete random variable x in X, suppose there are two discrete probability distributions p(x) and q(x) on this same space obtained from two classifiers, i.e., p(x) + q(x) = 1 and p(x), q(x) > 0. Then the KLD (discrete form) from q(x) to p(x) is given as in Equation 9 .

Since D KL (p(x)||q(x)) D KL (q(x)||p(x)), the symmetrical statistical divergence known as JSD has been proposed, which is derived from the KLD, and is a smoothed version of it. For probability distributions P = p(x) and Q = q(x) on the probability space X, suppose we have another probability distribution M = 1 2 (P + Q), then the discrete form of JSD is given as in Equation 10 .

The KLD and JSD values between every pair of classifiers from the four pre-trained CNN models are shown in Table 5 .

The proposed approach is compared with some standard deep transfer learning models, including the ones used for the ensemble in the present study, and the results thus obtained are tabulated in Table 6 . DenseNet161 performs closest to the proposed ensemble model, obtaining an accuracy of 96.91%, but still, the performance is 2.2% lower than the proposed method.

The results obtained by using different ensemble techniques on the SARS-COV-2 dataset [54] are shown in Table 7 . The fuzzy measures used for the Sugeno Integral has been found through extensive experimentation to be {0. 35 Among the explored ensemble techniques, the weighted average approach gives the result closest to the fuzzy integral ensemble, where the weights have also been assigned experimentally. This weighted average ensemble is a static computation process since there is no scope for dynamically refactoring the weights at the prediction time. Fuzzy integral fusion, on the other hand, can use the confidence scores of each classifier and tailors the weights on the fly, which is a different set for each sample data point. This enables further refinement of predictions even after setting the fuzzy measures.

Existing approaches for the chest CT classification for COVID-19 detection on the dataset used in this study mostly use a single CNN model for classification. Halder et al. [17] and Jaiswal et al. [25] used transfer learning with DenseNet201, for classification. Panwar et al. [44] used transfer learning with VGG-19 along with adding 5 more fully connected layers before the final classification layer. In their experiments on chest X-Ray images, they employed a Grad-CAM based colour visualization approach for interpreting the X-ray scans for further diagnosis. Angelov et al. [4] used the GoogLeNet architecture (non-pretrained model) for extracting deep features, which were used by an MLP classifier for the final classification on the chest CT-images dataset and Sen et al. [49] used a bi-stage deep feature selection approach for the COVID-19 detection problem. The results obtained by these methods on the SARS-COV-2 dataset [54] , used in the current study, have been compared with the model proposed by us in Table 8 . The performance of the proposed model is seen to be appreciably better than the existing methods in the literature. As compared to the single CNN model-based approaches by Jaiswal et al. [25] , Panwar et al. [44] and Angelov et al. [4] , the proposed approach performs significantly better justifying the use of an ensemble framework.

Gradient-weighted Class Activation Maps or GradCAM [48] provides a visual representation of the regions in an image that is focused on by the CNN model. The features from the convolution layers of the trained CNN model are used to produce a gradient-based weighted activation map which is superimposed on the input image to highlight the regions that are considered discerning by the CNN model. Figure 13 shows the GradCAM activations by the four different models used to form the ensemble on some sample images taken from the SARS-COV-2 dataset [54] . As seen from the figure, different models focus on different parts of the same chest CT image. For example, Figure 13 (a) shows a CT image of a COVID-19 infected patient. Figure 13(b) , (c), (d) and (e) respectively show the activations produced by VGG-11, GoogLeNet, SqueezeNet v1.1 and Wide ResNet-50-2. VGG-11 focuses on the lower part of the right lung (left side of the image) and the middle part of the left lung. GoogLeNet focuses on the middle to lower part of the left lung. SqueezeNet v1.1 focuses their attention on the GGO appearing on the upper and lower regions of the right lung and middle of the left lung, and Wide ResNet-50-2 focuses on the entirety of the right lung (primarily on the lower side) where the GGO is more concentrated.

The different models focusing on the different regions of the chest CT images justify that complementary information is being extracted, making the ensemble successful. From the examples shown, it is clear that all the models focus on different regions inside the lungs of the patients. Since GGO starts appearing in the lungs of COVID-19 infected patients from the onset of the disease, the models can classify the images correctly most of the time by looking for the white opacities in the lungs. Thus, the proposed framework is fit for detecting the early stages of COVID-19, unlike the rapid antigen test which is only effective after several days of the infection. This makes the proposed framework a good COVID-19 predictor for use in the practical field.

The proposed framework works reliably well for even heavily noisy images examples of which are shown in Figure 14 (a) and (f). Both the images in the figure have been taken in bad imaging conditions, the images being hazy and having improper contrast. In the CT-scan image shown in Figure 14 (f), both the lungs of the patient are not prominent, but even so, the proposed approach could classify the images correctly. For further justification, the GradCAM activations by the four base learners have been shown along with the images. In both cases, the models were able to focus on the correct regions inside the lungs to compute the predictions. And since different regions were focused on the different models, the ensemble was able to incorporate the complementary information provided by each.

However, Figure 15 (a) and (f) shows two misclassified samples from each category-COVID and Non-COVID. Figure 15(a) shows the lung CT-scan of a COVID-19 positive patient but has been classified as a Non-COVID case by the proposed model. The image when examined can be seen that does not prominently have GGO in either lung and possibly is a very mildly affected COVID case. The image has most likely been taken in a very early stage, where the COVID-19 virus has not even started affecting the lungs, leading to a wrong prediction. The GradCAM activations in Figure 15 (b)-(e) shows that the base learners were not able to localize on the correct regions of the lungs to extract discerning information. Figure 15 (f) shows the lung CT-scan image of a Non-COVID category patient which has been misclassified to be a COVID case by the proposed model. The bottom portion of the right (image perspective) lung has an erroneous white region, which might have been considered to be the GGO by the model (which is seen in 14 J o u r n a l P r e -p r o o f the GradCAM activation maps in Figure 15 (g)-(j)), resulting in a wrong prediction. Even so, only a few images have been misclassified by the proposed model, and the false positive rate is also low, which is important for a COVID classifier. Although the model fails to accurately classify all the samples provided, it still performs reliably well and is very sensitive to changes in chest CT scans.

COVID-19 is an extremely contagious disease whose alarming spread on the global scale has crippled the world health care systems. It has set back the global economy to a near stand-still as the countries of the world impose lockdown throughout the country to try and halt the spread of the virus. As the human toll increases, the need for a conclusive solution becomes direr. The primary reason for the rapid spread of the virus is the lack of equipment for population-wide screening of infected patients. Coupled with this, the gold-standard RT-PCR test is not all-reliable either, being only 71% sensitive, and takes 6-9 hours for a single test as well. This demands a fully automated system that can detect COVID-19 through other modalities in a fast and reliable manner.

In this paper, we have proposed a fully automated, computerbased detection method that uses lung CT-scan images for the classification of COVID-19 infected patients. Chest CT scans are more sensitive than RT-PCR tests and X-Rays, while also being more widely available than RT-PCR test kits. The proposed method uses deep learning techniques along with a fuzzy measure based integral, the Sugeno Integral for ensembling the features of four pre-trained deep transfer learning models.

The results from the proposed approach on the SARS-COV-2 dataset [54] outperforms the current state-of-the-art results on the same dataset. From the confusion matrix in Figure 12 , it is clear that the False Positive samples are very less which is important as explained in Section 4.2. Also, from Table 4 , it is evident that the accuracy and sensitivity are very high (compare with the 71% sensitivity of RT-PCR). So, it can be concluded that the proposed approach is reliable and can be used for COVID-19 detection.

As discussed before, the proposed model was unable to classify a few instances of mild COVID situations. Since the COVID-19 infection only affects the inside of the lungs, and there are no pleural effusions, a patch-based approach that selects patches from the image for training, or an attention mechanism can be developed in future. Choosing the fuzzy measures is an optimization problem, but for this study, we set the measures through extensive experiments. We aim to address this problem in future as well.

Multi-task deep learning based ct imaging analysis for covid-19 pneumonia: Classification and segmentation

Explainable-by-design approach for covid-19 classification via ct-scan. medRxiv

Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks

Application of deep learning technique to manage covid-19 in routine clinical practice using ct images: Results of 10 convolutional neural networks

Harris hawks optimisation with simulated annealing as a deep feature selection method for screening of covid-19 ct-scans

Prediction of covid-19 from chest ct images using an ensemble of deep learning models

Explainable deep learning for pulmonary disease and coronavirus covid-19 detection from x-rays

Prediction of respiratory decompensation in covid-19 patients using machine learning: The ready trial

Chest ct features of covid-19 in rome

An approach to the classification of covid-19 based on ct scans using convolutional features and genetic algorithms

Imagenet: A large-scale hierarchical image database

Chest ct accuracy in diagnosing covid-19 during the peak of the italian epidemic: A retrospective correlation with rt-pcr testing and analysis of discordant cases

Supervised learning from incomplete data via an em approach

Rapid ai development cycle for the coronavirus (covid-19) pandemic: Initial results for automated detection & patient monitoring using deep learning ct image analysis

Covid-19 detection from lung ct-scan images using transfer learning approach

Eie: efficient inference engine on compressed deep neural network

Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding

Deep residual learning for image recognition

Systematic investigation into generalization of covid-19 ct deep learning models with gabor ensemble for lung involvement scoring

Weakly supervised deep learning for covid-19 infection detection and classification from ct images

Convolutional networks with dense connectivity. IEEE Transactions on Pattern Analysis and Machine Intelligence

Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size

Classification of the covid-19 infected patients using densenet201 based deep transfer learning

Imagenet classification with deep convolutional neural networks

On information and sufficiency. The annals of mathematical statistics

Fuzzy rank-based fusion of cnn models using gompertz function for screening covid-19 ct-scans

Pneumonia detection in chest x-ray images using an ensemble of deep learning models

Et-net: an ensemble of transfer learning models for prediction of covid-19 infection through chest ct-scan images

Backpropagation applied to handwritten zip code recognition

Artificial intelligence distinguishes covid-19 from community acquired pneumonia on chest ct

Coronavirus disease 2019 (covid-19): role of chest ct in diagnosis and management

Development and clinical application of a rapid igm-igg combined antibody test for sars-cov-2 infection diagnosis

Network in network

Machinery fault diagnosis based on fuzzy measure and fuzzy integral data fusion techniques. Mechanical Systems and Signal Processing

Ergodic theory and virtual groups

Covxnet: A multidilation convolutional neural network for automatic covid-19 and other pneumonia detection from chest x-ray images with transferable multireceptive feature optimization

Computer-assisted decision support system in pulmonary cancer detection and stage classification on ct images

The jensenshannon divergence

A deep learning approach to characterize 2019 coronavirus disease (covid-19) pneumonia in chest ct images

Automated detection of covid-19 cases using deep neural networks with x-ray images

Time course of lung changes on chest ct during recovery from 2019 novel coronavirus

A deep learning and grad-cam based color visualization approach for fast detection of covid-19 cases using chest x-ray and ct-scan images

Deep bidirectional classification model for covid-19 disease infected patients

The fuzzy integral

Coronavirus pandemic (covid-19)

Grad-cam: Visual explanations from deep networks via gradient-based localization

A bi-stage feature selection approach for covid-19 prediction using chest ct images

Lung infection quantification of covid-19 in ct images with deep learning

Large-scale screening of covid-19 from community acquired pneumonia using infection size-aware classification

Covid-19 detection in ct images with deep learning: A votingbased scheme and cross-datasets analysis

Very deep convolutional networks for large-scale image recognition

Sars-cov-2 ct-scan dataset: A large dataset of real patients ct scans for sars-cov-2 identification. medRxiv

Fuzzy measures and fuzzy integrals-a survey

Pseudo-additive measures and integrals

Going deeper with convolutions

Rethinking the inception architecture for computer vision

Real-time rt-pcr in covid-19 detection: issues affecting the results

Information fusion in computer vision using the fuzzy integral

The interpretation of fuzzy integrals and their application to fuzzy systems

Fully automated unified prognosis of covid-19 chest x-ray/ct scan images using deep covix-net model

Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images

Contrastive cross-site learning with redesigned net for covid-19 ct classification

Fuzzy integral with particle swarm optimization for a motor-imagery-based brain-computer interface

A deep learning system to screen novel coronavirus disease 2019 pneumonia. Engineering

Covid-19 science report: diagnostics. ScholarBank@NUS

Covid ct-net: Predicting covid-19 from chest ct images using attentional convolutional network

Wide residual networks

We would like to thank the Centre for Microprocessor Applications for Training, Education and Research (CMATER) research laboratory of the Computer Science and Engineering Department, Jadavpur University, Kolkata, India for providing us with the infrastructural support.

The authors declare to have no conflicts of interest.[1] Ai, T., Yang, Z., Hou, H., Zhan, C., Chen, C., Lv, W., Tao, Q., Sun, Z.,

J o u r n a l P r e -p r o o f

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us.We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property.We understand that the Corresponding Author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). He/she is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible by the Corresponding Author.