key: cord-0105032-27tqj46o
authors: Liu, Boyi; Yan, Bingjie; Zhou, Yize; Yang, Yifan; Zhang, Yixian
title: Experiments of Federated Learning for COVID-19 Chest X-ray Images
date: 2020-07-05
journal: nan
DOI: nan
sha: d3e2036d7ec2cdb61a33f0ffebbce184ac85aa66
doc_id: 105032
cord_uid: 27tqj46o

AI plays an important role in COVID-19 identification. Computer vision and deep learning techniques can assist in determining COVID-19 infection with Chest X-ray Images. However, for the protection and respect of the privacy of patients, the hospital's specific medical-related data did not allow leakage and sharing without permission. Collecting such training data was a major challenge. To a certain extent, this has caused a lack of sufficient data samples when performing deep learning approaches to detect COVID-19. Federated Learning is an available way to address this issue. It can effectively address the issue of data silos and get a shared model without obtaining local data. In the work, we propose the use of federated learning for COVID-19 data training and deploy experiments to verify the effectiveness. And we also compare performances of four popular models (MobileNet, ResNet18, MoblieNet, and COVID-Net) with the federated learning framework and without the framework. This work aims to inspire more researches on federated learning about COVID-19.

The COVID-19 pandemic has caused continuous damage to the health and normal production of people all over the world. Therefore, researches on detecting and diagnosing COVID-19 patients are very meaningful [1] [2] . The clinical manifestations of COVID-19 infected pneumonia are mainly fever, chills, dry cough, and systemic pain. A few patients have abdominal symptoms. It is worth noting that there are asymptomatic patients in the population. So it is necessary to test more people as soon as possible. A key step in judging and treating COVID-19 is the effective screening of infected patients. One of the key screening methods is the use of chest X-rays for radiology. Computer vision and machine learning technology play an important role in this approach. At present, artificial intelligence, especially deep learning, has become an important technology for computer-aided medical applications and has achieved remarkable results in medical imaging. Deep learning has made a huge contribution to the classification of chest X-ray radiology in the medical field, and it has become an effective tool for doctors to judge and analyze the condition. To obtain an accurate and robust depth model, the core element is large and widely diverse training data. However, out of the protection and respect of the privacy of patients, the hospital's specific medical-related data did not allow leakage and public research. Collecting such training data was a major challenge. To a certain extent, this has caused a lack of sufficient data samples ¼ ¼ when performing deep learning approaches to detect COVID-19. Federated Learning is an available way to address this issue. It can effectively address the issue of data silos and get a shared model without obtaining local data. In the paper, we firstly propose the use of federated learning for COVID-19 data training and deploy experiments to verify the effectiveness.

Federated learning is a framework of learning across multiple institutions without sharing patient data. It has the potential to fundamentally solve the problems of data privacy and data silos. Applications of federated learning in medical big data are promising researches. Federated learning is capable of utilizing the non-shared data from different hospitals, enlarging the sample size of the model training, and improving the accuracy of the model. The core of federated learning is to use data sets distributed on multiple devices to jointly build a shared model and does not require local raw data sharing. This precisely protects patient data. In the case that COVID-19 medical imaging data is still distributed in various countries and hospitals, federated learning experiments for medical images of COVID-19 that conducted in this work are necessary.

In this work, we conducted four individual experiments to present the performances in federated learning of four different networks for COVID-19 CXR images: CovidNet [3] , ResNeXt [4] , MobileNet-v2 [5] , and ResNet18 [6] . Further, we analyzed the results and proposed possible future improvements to inspire more research in federated learning for COVID-19.

With the continuous development of the new coronavirus epidemic, more and more researchers are committed to join the ranks of fighting the epidemic through AI-related technologies. Researchers use AI to make it play a role in the epidemic. A series of recent studies on COVID-19 medical imaging analysis and diagnosis have sprung up. These studies have completed the diagnosis of COVID-19 based on medical imaging technology. We use the current CT scanning technology to complete the image generation of the special radiological features and image modes of COVID-19. After that, the researchers used machine learning methods to classify and recognize the images generated during the CT scan diagnosis process. This method greatly reduces the workload of medical staff, and at the same time plays a role in assisting doctors in diagnosing the pathological characteristics of patients. At present, many studies have targeted disease as a binary classification problem, that is, "health" and "positive new coronavirus". Next, we introduce the application of COVID-Net in COVID-19 image classification and recognition and complete diagnosis of pathological features.

COVID-Net specifically proposes a neural network that uses PEPX compression network structure to identify COVID-19 pneumonia CXR images. At the same time it retains the performance of the network to a great extent and is highly sensitive to the pneumonia characteristics of COVID-19. Based on the advantages of CXR imaging for rapid triage of COVID-19 screening, availability, ubiquity, and portability, they make predictions through the COVID-Net interpretability method. This allows us to not only gain a deeper understanding of the key factors associated with COVID cases. This can help clinicians perform better screening. We can also review COVID-Net, a method based on CXR images to verify that it is making decisions.

Federated learning is a recent emerging research that has been extensively studied in the fields of financial security, artificial intelligence, and robotics [7] [8] [9] . The training data will be distributed on each mobile device, not all of them will be sent to the central server, and only the updated data on each device will be aggregated to the central server. After joint optimization, the central server returns to the global state of each device, and continues to accept the updated data calculated by each client in the new global state. This method is Federated Learning [10] . Federated Learning or Federated Machine Learning [11] can solve the problem of unprotected large-scale private data and complete updating learning of devices without exchanging large amounts of data. This decentralized training model approach provides privacy, security, regulation, and economic benefits [12] . Federated Learning presents new statistical and system challenges when training machine models on distributed device networks [13] . Federated Learning, which relies on scattered data, brings many aspects of research: Fei Chen et al.

identified the combination of Federated Learning and Metalearning as a major advance in Federated Learning [14] . Konstantin Sozinov et al have made some progress in applying Federated Learning to human activity identification [15] .

In this section, we provide a comprehensive overview of federated learning. Furthermore, the definition, architecture, training process and parameters update method of the federated learning system [5] for COVID-19 CXR images are considered.

In the work, we define N COVID-19 CXR images owners as F 1 , F 2 , F N . We assume that they are from different hospitals. Patient medical data is not allowed to be shared, including CXR images. Under this constraint, all of them want to train their own model by merging their respective data D 1 , D 2 D N . A conventional method exists to put all the data together and use D = D 1 D 2 D N to train to get a model M S UM. Federated learning is a systematic learning process. In this process, the data owners jointly train the model M F ED. During this process, any data owner F i will not disclose their own data D i to others. In addition to this, the accuracy of M F ED represented as V F ED should be very close to the V S UM performance of M S UM. In the form of expression, let ε be a non-negative real number; If |V F ED−V S UM| < ε, we can think that the federated learning algorithm has a ε error accuracy.

In this part, we will introduce the basic framework of federated learning, the training structure, and the way to update the parameters. Federated learning is a distributed learning method. The server is used to maintain the overall main model and distribute it to various user terminals. For privacy issues, users train learning models on local terminals. The server will set the score S, and extract the user terminal according to the proportion to update the central model of the server. Then upload the user-improved model parameters to the server to update the server model parameters. Subsequently, it is distributed to user terminals to improve the user terminal model. In this way, we continue to improve the central model of the server and the local model of the user terminal. This approach is capable of ensuring the accuracy and privacy of the user terminal, utilize the user terminal's computing power and a large amount of user data to learn, and maintain an excellent central model.

In the FL training system, the owner of the data acts as a participant in the FL process. And they jointly train the machine learning (ML) model of the aggregation server center. In this control architecture, a basic premise assumption is that the data owners are honest and the data they provide is true. This requires data users to use their real private data for training and submit the trained local model to the FL server.

Generally, the FL training process includes the following three training steps. We first define that the local model refers to the model trained on each participating device, and the global model refers to the model after the FL server has been aggregated.

• Step 1: Implements task initialization. The server determines the training task, which is to determine the target application and corresponding data requirements. At the same time, the server specifies the global model and establishes parameters during the training process, such as the learning rate. Afterwards, the server allocates the initialized global model W G 0 and training tasks to the participating clients to complete the task allocation. • Step 2: Implements the training and update of the local model. The training is carried out on the basis of the global model W t G, where t represents the current iteration index, and each participating user uses local data and equipment to update the local model parameters W t i . The final goal of participant i in iteration t is to find the optimal parameter W t i that minimizes the loss function L(W t i ). • Step 3: Realizing the aggregation and update of the global model. The server aggregates the local models of the participating users and sends the updated global model parameters W ( t + 1) G to the users who hold the data.

In this session, we will explain our experiments on the recognition of COVID-19 pneumonia CXR images using various models and federated learning frameworks.

The dataset used to train and evaluate model is COVIDx dataset, which is one of the open access dataset with the largest number of COVID-19 pneumonia CXR images. It contains covid chestxray dataset, COVID-19 Chest X-ray Dataset, Actualmed COVID-19 Chest X-ray Dataset, covid-19 radiography dataset which is public on kaggle and RSNA Pneumonia Detection Challenges dataset. There are 15,282 images in this dataset, including 13,703 images for training and 1,579 images for testing. There are three kinds of labels in the dataset. They are Normal (which is asymptomatic), pneumonia (which is non-COVID19 pneumonia) and COVID19 (which is pneumonia caused by novel coronavirus). The various data distributions are shown in Figure  1 .

B. Experimental Settings 1) Model: The main task in the experiment is image classification. There are already many classic neural networks in this field, and there are models specifically designed for the recognition of COVID-19 pneumonia CXR images. Four models are used in the experiments.

• COVID-Net: A neural network specifically proposed to identify COVID-19 pneumonia CXR images utilizes PEPX to compress the network structure while preserving the network's performance to a large extent. At ing split-transform-merge strategy to convert singlecore convolution into multi-core convolution, but the topology is the same as ResNet18. • MobileNet-v2: It is a lightweight convolutional neural network. Unlike residual neural network, residual neural network uses a convolution kernel to first compress and extract features and then expand, but it expands and extracts more features and then compresses. 2) Implementation and Training: Federated learning is a pseudo-distributed training completed on one machine in our experiment. For each agent, there is a separate model, which is reset to the updated central model after each central model update. The models are all implemented by PyTorch, and the training set and test set images are resized to (224,224) for model training. Each terminal agent uses the Adam optimizer with learning rate = 2e-5 and weight decay = 1e-7. The framework for federated learning is trained under the GPU acceleration of NVIDIA Tesla V100 (32GB) on Ubuntu 18.04 system. Other training-related parameters are shown in Table 1 .

During the experiment, the loss of several models can converge, and the decline of loss during the training process is shown in Fig 3. After using the same parameter training four models for 100 rounds, ResNet18 has the fastest convergence speed, and the highest accuracy rate (96.15%, 91.26%) on both the training set and the testing set. The ResNeXt convergence rate is closely followed, but the accuracy rate is not as good To explore the sensitivity of the models to each label, we counted the accuracy of each model for each label, and the performance results of the models are shown in Fig.4 and Tabel 3.

At the same time, we compared the training results without the federated learning framework with those using the federated learning framework. Because round=100 and local epoch=3 in the federated learning parameters, we set epoch=300 when training the model separately, so as to compare with the loss convergence during the federated learning training process, as shown in Fig.5 . It was found that the loss convergence rate caused by the use of federated learning decreased slightly. The result of training accuracy of a single network is shown in Table 4 . As presented in Fig.5 , we compared the training procedure based on federated learning with the training procedure without federated learning. MobileNet-V2 has a larger accuracy gap between the FL based approach and the individual approach while ResNet18 has a smaller accuracy gap. If considering the number of parameters at the same time, Mo-bileNet and ResNet18 have a higher performance. MoblieNet has the fewest parameters and the lowest accuracy. ResNet18 has the second fewest parameters and the highest accuracy.

We used the Grad-CAM++ method to perform Visual Explanations on the models, and the results are shown in Fig 6-8 . From left to right in each figure are the COVID-Net, ResNet18, ResNeXt, and MobileNet-v2 models. The first line is the result of training using the federated learning framework, and the second line is the result of training without the federated learning framework.

In the work, we conducted experiments on COVID-19 identification with CXR images based on the federated learning framework. We conducted CXR images trainning with four different models: MobileNet, ResNet18, MoblieNet and COVID-Net, and comparison experiment between training with federated learning framework and training without federated learning framework.The experimental results show that ResNet18 has the best performance both in training with FL and without FL. ResNeXt has the best performance in images with COVID-19 labels. MoblieNet has the fewest number of parameters. Therefore, the work indicates that ResNeXt and ResNet18 are better chosen for COVID-19 identification among the four popular models.

An improved method of covid-19 case fitting and prediction based on lstm

Covid-19 public opinion and emotion monitoring system based on time series thermal new word mining

Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest xray images

Spatial image steganalysis based on resnext

Mo-bilenetv2: Inverted residuals and linear bottlenecks

Medical image retrieval using resnet-18

Lifelong federated reinforcement learning: a learning architecture for navigation in cloud robotic systems

Federated imitation learning: A novel framework for cloud robotic systems with heterogeneous sensor data

Design and implementation of a novel precision irrigation robot based on an intelligent path planning algorithm

Communication-efficient learning of deep networks from decentralized data

Federated learning: Strategies for improving communication efficiency

Federated learning with non-iid data

Federated multi-task learning

Federated meta-learning for recommendation

Human activity recognition using federated learning