key: cord-0691391-p69p1d5b
authors: Nasser, Nidal; Fadlullah, Zubair Md; Fouda, Mostafa M.; Ali, Asmaa; Imran, Muhammad
title: A lightweight federated learning based privacy preserving B5G pandemic response network using unmanned aerial vehicles: A proof-of-concept
date: 2021-12-20
journal: Computer Networks: the International Journal of Distributed Informatique
DOI: 10.1016/j.comnet.2021.108672
sha: 7bdf5a6e15689b39ac2ff6b58e8d4df87dccb318
doc_id: 691391
cord_uid: p69p1d5b

The concept of an intelligent pandemic response network is gaining momentum during the current novel coronavirus disease (COVID-19) era. A heterogeneous communication architecture is essential to facilitate collaborative and intelligent medical analytics in the fifth generation and beyond (B5G) networks to intelligently learn and disseminate pandemic-related information and diagnostic results. However, such a technique raises privacy issues pertaining to the health data of the patients. In this paper, we envision a privacy-preserving pandemic response network using a proof-of-concept, aerial-terrestrial network system serving mobile user entities/equipment (UEs). By leveraging the unmanned aerial vehicles (UAVs), a lightweight federated learning model is proposed to collaboratively yet privately learn medical (e.g., COVID-19) symptoms with high accuracy using the data collected by individual UEs using ambient sensors and wearable devices. An asynchronous weight updating technique is introduced in federated learning to avoid redundant learning and save precious networking as well as computing resources of the UAVs/UEs. A use-case where an Artificial Intelligence (AI)-based model is employed for COVID-19 detection from radiograph images is presented to demonstrate the effectiveness of our proposed approach.

Recently, due to the devastating spread of the novel coronavirus (COVID- 19 ) on a global scale [1, 2] , researchers started exploring how to formulate intelligent pandemic response networks [3, 4] . Communication networks, particularly unmanned aerial vehicle (UAV)-integrated fifth generation and beyond (B5G) mobile networks, can be leveraged to construct agile pandemic response networks to disseminate pandemic recommendations, facilitate digital contact tracing, monitor the spread of the pandemic, and so forth. Incorporating intelligence into the UAVs using Artificial Intelligence (AI) techniques is, however, posing computational and privacy challenges. The surge of pandemic-related traffic will also lead to increasing the communication delay and overwhelm the backhaul links. Therefore, to construct an efficient UAV-based pandemic response network to complement the terrestrial cellular infrastructure for enhanced service coverage, collaborative learning by the UAVs [5] in a lightweight, as well as privacy-preserving manner, is essential.

Even though the drone resources for processing, memory, and battery have improved much recently, the UAVs are anticipated to still lack, in B5G networks, the resources to execute intelligent and coordinated learning based on the monitored and collected pandemic data of mobile users (UEs). Because maintaining privacy is a big concern for the acquired health data of the UEs, existing methods using explicit user reporting on their locations and pandemic data may not be directly adopted. It is worth clarifying the difference between the security and privacy of patient data. In the context of this paper, security typically implies safeguarding the data of the users/patients against unauthorized access of the data. Security measures can be enforced by the state-of-the-art cryptography to limit which stakeholders may have access to the information. On the other hand, the term privacy is harder to define since user-specific details (e.g., user/patient identity and other sensitive data) can also be secure data. In the context of our work, privacy focuses upon masking the user/patient-specific information. Thus, when the patients' devices, referred to as UEs, communicate with the B5G terrestrial/aerial base stations (gNBs) and provide their raw radiograph images for pandemic data analytics, there are scopes for mapping back the patient data to individual users and reveal their identities, whereabouts, living patterns, and so forth. The cloud-based analytics could also reveal certain information pertaining to the identification of the users [6] . The idea of this paper is to use an asynchronously updating federated learning framework to thwart such threats whereby the wisdom of the patients' radiograph data are shared rather than the explicit images to train a central AI model without exposing the user-specific details in B5G networks. In this paper, by addressing the privacy challenge, we propose a collaborative learning method exploiting two-layer federated learning [7] . In our proposed technique, the general and specific weight parameters of the AI models locally learned in two layers (UEs and UAVs) are asynchronously updated to update the global AI model at the cloud, managed by hospitals and public health authorities. While reducing B5G network overheads, the proposed technique still allows the cloud to predict the appropriate pandemic features with reasonably high accuracy without the need to receive raw health data, locations, and other information of the UEs.

The key contributions of our work in this paper are outlined below.

1. To provide a distributed privacy-preserving data collection and analytics framework at the same time, a federated learning-based framework is presented. This facilitates the exchange of wisdom of the underlying B5G mobile user data instead of explicitly sharing the raw data (e.g., location, mobility, health status, and so forth) with the central cloud. 2. Asynchronous update of weights of local models is considered in the envisioned federated learning framework to significantly improve the communication overheads. This facilitates lightweight decentralized model construction and execution at the resource-constrained mobile UEs and other distributed/local nodes (LNs). 3. The need to seamlessly merge the asynchronously updating federated learning framework with the blockchain-enabled centralized aggregator is identified as an open research issue that will stimulate much work in the area of privacy-preservation of big medical data. 

Federated learning technique and scenario B5G-ready? [14, 8, 12, 18, 20, 22] Horizontal, vertical, and hybrid data partitioning and/or cryptographic, perturbative, and anonymization techniques for privacy

No [16, 17, 13, 15, 9, 21, 11] Specific focus on privacy-preservation in federated learning setups without any wireless communication parameter

No [22] Privacy leakage scenarios with external and external attackers, active/passive attacks, inference attacks

No [24] Temporally asynchronous weight update of federated learning No [19, 23] Federated machine learning to address energy, bandwidth, delay and data privacy concerns in wireless communications by performing decentralized model training Yes ized to revolutionize the smart healthcare landscape recently [25] due to its potential in bypassing the need for centralizing or releasing patient data and raising regulatory, ethical, legal, and even technical challenges. However, the application of federated learning for combating COVID-19 was not considered in this work. The work in [26] raised the importance of respecting and protecting patients' privacy while collecting their health data for COVID-19 identification. Their work used federated learning to train COVID-19 models and compared performances with popular AI models, including ResNet18, MobileNet, and COVID-Net. These models were also compared with a similar federated learning model presented in [27] , focusing on chest X-ray and CT (computed tomography) scan images with normal and COVID-19 samples. A recent endeavor from NVIDIA attempted to construct AI models to predict oxygen needs without the need to share data and devise a generalizable model regardless of geographical location, patient population or data size [28] . The work in [29] identified the generalizability issue in building AI models to gain trust from both patients and clinicians and implemented the 5 unified CT-COVID AI diagnostic initiative (UCADI) framework for federated learning. This initiative allows any research institution or hospital with data and computational resources to contribute their local model parameters to enrich the centralized learning framework's performance. In this way, the data heterogeneity problem, a key challenge in AI diagnostics in terms of patient diversity, diagnostic image resolution, number of cases, and so forth, is also implicitly considered. In this way, the participants worldwide using UCADI are anticipated to assist health practitioners in detecting COVID-19 patients robustly and reliably. However, the aforementioned methods share an inherent shortcoming in both the system model used and the manner in which the AI models are shared, which restrict the performance of the federated learning technique. For instance, using these methods, the patients may not be able to preserve their privacy because of the need to explicitly share their raw data with hospitals or clinics, which are assumed to secure and perform the federated learning on behalf of the patients. On the other hand, the distributed/transfer learning methods [30] also lead to an exchange of content-specific data (i.e., health data) since they require the UEs to collaborate. Furthermore, the existing applications of federated learning to address the COVID-19 pandemic are yet to explore whether an asynchronous model updating paradigm could be used to boost the learning performance with much-improved network overhead. To mitigate the UEs' privacy concern and significantly reduce network overhead, a novel proactive method is urgently required in aerial as well as terrestrial base stations.

Fig. 1 summarizes our considered B5G network scenario, which comprises an integrated aerial and terrestrial network. Several UAVs act as aerial base stations to complement the terrestrial base stations (gNBs) to collaboratively serve user equipment/entities or UEs with an improved service coverage. Typically the UEs represent mobile users such as patients with smartphones, wearables, fitness trackers, and other smart devices monitoring health. For simplicity, in the remainder of the paper, the term UE will refer to usersmartphones capable of training local AI models and establishing an uplink with the aerial/terrestrial base stations to exchange the AI model parameters. Further exploiting this scenario and based on our earlier work in [31] , the system model is envisaged as a software defined network (SDN) which manages the data, control, and application planes, as depicted in Fig. 2 ing this approach, the SDN controller may virtualize network management algorithms as applications on commodity hardware having various computing resources, such as centralized processing units (CPUs), graphics processing units (GPUs), Baseband Unit (BBUs) pool, and other computing resources to carry out the computation of routing, link scheduling, and other network management tasks for terrestrial and aerial base stations. To summarize, this system model is considered to efficiently manage the integrated aerial and terrestrial links. The SDN can manage the deployment, link quality, and topologies of the UAVs to permit them to adaptively operate in heterogeneous radio access technology (RAT) environments. Thus, the network management is quite flexible and can be easily upgraded simply by updating the corresponding applications. Furthermore, to enhance the usage of the computing resources, optimal caching placement can be done on the drones to store AI models received from many UEs and carry out their own learn-J o u r n a l P r e -p r o o f ing to collaborate with other UAVs in learning pandemic features, and thus provide an agile pandemic recommendation and response network.

The considered communication system model is delineated from hereon based on [32] . During every transmission time round, UE-side local training model parameters are transmitted to the centralized aggregator via terrestrial base stations and/or UAVs to converge to an accurate AI model in a distributed manner. The improved AI model is then periodically shared with the UEs. For each time round, uplink and downlink transmission overheads considering channels with Additive White Gaussian Noise (AWGN) are assumed [33, 34] . The communication rounds are assumed to continue until the global model converges with a reasonable performance measure. Based 8 on the radiograph images collected, the m most relevant images are selected by the centralized aggregator for caching at the serving terrestrial base stations and UAVs. Let the UE, terrestrial base station, and UAVs represent the distributed or local computing nodes, referred to as local nodes (LNs) for brevity. Let d i denote the transmission rate of UE i , which can be estimated as:

where B, P t i , g i , and ζ denote the transmission bandwidth of the serving LN, transmission power of UE i , channel gain of UE i , and background noise power, respectively. We assume the same radiograph data-size, θ for all UEs. Then, the transmission time, T i T x , of a local model update with this data-size is expressed as:

Then, the time taken for one global iteration by UE i , denoted by T i G is given by:

where ϵ i refers to the local accuracy while T i L denotes the computation time of a local iteration which will be described later. The caching entities of LNs are assumed to have finite storage. The LNs use their local learning models to make intelligent decisions on which radiographs should be cached. The cache is assumed to be able to store m files of a constant size. Based on its local data, each LN is assumed to be able to independently calculate an update to the current global learning model and exchange the update to the centralized aggregator to aggregate a new global model. Each UE also trains a local dataset, which is generated from the radiographs, and can be co-trained with other structured data including different current activities, current location, contextual information derived from personal settings of mobile device sensors, and so forth. It is worth noting that each LN, particularly each UE, may have an individual context space. The local learning model in each UE and LN is based on Convolutional Neural Network (CNN) which will be described in Sec. 4 . The local learning model update of UE i is impacted by the quality of its locally available data that can be expressed as the local accuracy, ϵ i . If the value of ϵ i is high, we need to perform fewer local (as well as global) iterations bounded by log( 1 ϵ i ). This can be used to predict the content popularity more precisely and quickly. Now, we describe J o u r n a l P r e -p r o o f Journal Pre-proof the computation time of a local iteration, denoted by T i L , in the local model training for UE i as follows:

where s i denotes the size of the local data of UE i . The CPU cycle and number of CPU cycles needed to conduct the local model training with the sample-size s i are denoted by f i and c i , respectively. In the scenario presented in the preceding section, the small cell UEs, which are served by a terrestrial or aerial base station, participate in pandemic learning such as COVID-19 data collection regarding the user's symptoms and vitals (e.g., temperature, heartbeat, blood oxygen saturation), travel history (e.g., through geolocation sensing), radiograph monitoring using emerging portable devices, and so forth. The users should not provide raw data to the aerial base stations or the cloud to avoid potential privacy breaches. In addition, the aerial base stations, despite their potential to extend B5G network coverage, are particularly useful for remote/rural locations and pandemic-affected sites where small cells are already prone to experiencing network congestion. Let us consider only those UAV nodes with sufficient energy budget during their flight. With the aforementioned scenario, the terrestrial and UAV base stations require a distributed methodology to learn pandemic features (e.g., COVID-19 symptoms, pneumonia classification using radiographs, and so forth) with privacy preservation to maximize the learning accuracy. Because the base stations lack a global view of UEs in various small cells, how they may leverage the proposed system model in Fig. 2 to perform decentralized machine learning without collecting the raw health data of the users arises a challenge. In this vein, the base stations collect the local training model from the UEs, and share this information with other base stations and then the cloud to further boost the decentralized training among UAVs. The cloud, upon receiving the locally trained models at the UE layer as well as the UAV layer, can then train a global model to arrive at a reasonable decision with significantly high accuracy. In other words, this is a two-layer federated learning problem. The objective is to minimize a loss function:

where f i (w, x k , y k ), S i , and S denote the loss function of UE i , the sample size (locally monitored/acquired health data) of UE i and the training J o u r n a l P r e -p r o o f Journal Pre-proof samples used by the cloud. Every UE consists of a weight vector, w, which represents its locally trained AI model parameters using x k and y k . The value of f i (w, x k , y k ) increases with growing prediction error (x ⊺ k w − y k ). Therefore, the constraint w 1 = w 2 = ... = w k is needed to guarantee the convergence of global learning such that all UEs, UAVs, and the cloud can eventually derive the same AI model without explicitly exchanging the raw health data of UEs. Furthermore, in the context of the pandemic, there are two types of UEs considered in the considered system model, namely the individual UE type and the organizational UE type. The individual UEs do not have the capability of collecting certain information (e.g., X-ray images as mentioned) with their simple device type, and in this case the collected information constitutes their location, movement, body temperature, blood-oxygen saturation, and so forth. On the other hand, the organizational UEs such as a caregiver's system (e.g., a hospital EMR system) or biomedical equipment with IoT capability are assumed to have the capability of sensing/acquiring other information such as X-ray images as presented in the performance evaluation scenario.

To summarize, the research challenge is to design a technique so that (1) each UE is capable of training a shared global model with its local model update based on its data, and (2) each UE shares its updated local model with the cloud through the aerial base stations to revise/update the global model. Similarly, the aerial base stations confront their own local model training problem to predict UEs' pandemic patterns and participate in second-level federated learning with the global model. Until the loss function is minimized and/or the global model accuracy is deemed reasonable, this distributed training process needs to continue. Therefore, how to design such a decentralized technique with reduced network overhead and privacypreservation property can be regarded as the main research problem of our work in this paper.

The fundamental idea of our proposal is to allow the UEs, constrained by small computational capacity, to train and build localized AI models based on the monitored health data of the user. The UEs participate in the first layer of federated learning to make this possible. Furthermore, the AI model parameter exchange of our proposed system, as shown in Fig. 2 Figure 3 : Proposed algorithm.

J o u r n a l P r e -p r o o f Journal Pre-proof proactive pandemic feature learning algorithm leveraging a second-layer federated learning with the collaboration of UAVs to acquire pandemic modelspecific information from the UEs of small cells when the terrestrial base stations are either congested or unable to provide service coverage to UEs due to geographical barriers (e.g., in rural/remote locations). To discover the hidden features from complex pandemic/health data, we utilize a customized convolutional neural network (CNN) to build the AI model at UEs, UAVs, and the cloud [35, 36] . For the UE-generated local AI models, shallow and dense layers are considered to be smartly scheduled for transmission to the serving aerial base station to reduce network overhead. This approach is different from the traditional federated averaging [7] , where the parameters of the entire deep neural network are updated at a single shot. This traditional model update process results in large communication overheads. On the other hand, an asynchronous parameter update methodology, inspired by the work in [37, 30] may deal with this problem as follows. The shallow layer (general) parameters acquire general features of the pandemic while the deep layer parameters, big in size, capture specific features regarding specific symptoms and context information of the users. Consequently, the shallow layer weights can be updated and exchanged more often compared to deep layer-parameters. This asynchronous weight updating methodology is shown in the algorithmic steps of Fig. 3 .

First, we discuss the local model update at UE i in Fig. 3 . The proposed algorithm has three inputs: the UE identifier i, the weight vector of the UEgenerated model w, and a control flag checkShallow. The collected health data of the pandemic and context information are split into mini-batches, and B represents the mini-batch size. ϵ denotes the epoch of the local model. The control flag checkShallow is exploited to select whether all the layers or the shallow layers will be updated. Stochastic Gradient Descent (SGD) is used w = w − η * ▽l(w; b). Here, η and b refer to the learning rate and minibatch, respectively. On the other hand, the aerial base station, UAV j , polls the UEs by initializing a clock, in synchronization with the UEs and the cloud. UAV j receives the timestamp for each of the UEs it serves. Then, UAV j starts a temporary buffer. Using the timestamp information, UAV j makes a decision to execute either the shallow or deep parameters update. Then, for each round, the UAV polls its served UEs (UE 1 to UE k where i is an index between 1 and k) and receives the relevant parameters from UE i given the current timestamp and the checkShallow control variable. Next, UAV j appends the parameters received from UE i to the buffer. After polling all the UEs, this buffer contains the information of shallow-only or complete weight vector parameters for the UEs served by that UAV. As shown in Fig. 3 , the shallow-only/deep parameters are sent during the flag switch interval, which is considered to be much smaller than the updated global model receiving interval of each UE. The content of the buffer is then sent to the cloud. UAV j also locally executes its deep learning structure for predicting pandemic trends in collaboration with the neighboring UAVs. Then, the UAV continues to poll the UAVs again in the same manner until the model accuracy becomes reasonable.

At the cloud-end, the temporally weighted aggregation occurs over multiple communication rounds [37] . The timestamps for general and specific weights/parameters are initialized. The federated learning takes place in each round T , which is split into ∆ time-slots. Before each round, the cloud exchanges the timestamp list it initialized for each UE to the serving base station. Then, for each round T , the time-slots which are in δ (the set of time-slots during which specific parameters are fetched from UEs), the control variable checkShallow is set to true. A participating subset of the clients is randomly selected per round. The specific and general parameters are fetched from each base station. Thus, the aggregation is performed to update the shallow and deep parameters (w g and w s , respectively).

Based on these steps, we present the two algorithms for our envisioned asynchronous federated learning model for privacy-preserving COVID-19 screening, namely Algorithms 1 and 2. The global and local AI models, constructed at the cloud and aerial base station/terrestrial base station/UE, respectively, are facilitated by either a custom ANN (artificial neural network) model or a customized CNN model developed in our earlier work to carry out a centralized COVID-19 prediction [38] .

First, we describe Algorithm 1 used for training the initial global AI model. This algorithm accepts the set of all available UAVs, terrestrial small cell base stations (TSCs), and UEs. The objective of the algorithm is to return M g , the trained global model. The algorithm parameters are initialized in step 1. The elements of the set of all the local or distributed nodes (i.e., the distributed learning agents in terms of the deployed UAVs, TSCs, and UEs), referred to as LN , is iterated through steps 3 to 5. Thus, the parameters obtained from the distributed nodes can be aggregated. The aggregated parameters are then shared with the distributed nodes by the cloud. Learning continues until the learning performance in terms of loss is improved compared to a pre-determined threshold, (min loss ), in step 7 of Algorithm 1. Next, Algorithm 2 is presented to execute the distributed learning on the LN's side. The inputs of the algorithm consist of u, ∆, T , and λ. Throughout the first three steps, the required parameters related to the algorithm and the local model (M u ) is set to selected values. Following this, a binary parameter, referred to as the contextual switch and denoted by cs u , is set to false. In the subsequent step, the data size of each radiograph data is evaluated by setting the instance and feature sizes as d r and d c , respectively. Then, the new data is loaded and standardized as:

where X u indicates the data after performing resizing,X u denotes the mean value of the data, and σ u refers to the standard deviation.

Step 9 of Algorithm 2 signifies that the training process at each LN takes place over t = 1 to t = T iterations, where T denotes the number of timesteps during which the whole process at each LN is executed once. After ∆ time rounds, when the condition for setting the switches on is verified in step 10, the contextual switch (cs u ) is set to true. Then, an alert is triggered Steps 16 to 21 are executed while flag cs u is true. Through these steps, the local weight (W u ) with shallow parameters and relevant access and time-step information (block u , timestep u ) are shared with the cloud. The parameter α refers to the rate of the deep parameters exchange. The parameter timestep u holds the iteration information when the deep parameter exchange with the cloud occurs. Through steps 20 and 21, cs u is set to false. This is done to avoid unauthorized transmissions. Furthermore, an on-display alert is generated at the LN for the LN manager to turn off the physical switch until the next time round commences. If cs u is false, the shallow parameters of (1 − α) ratio from the local model (M u ) are shared with the cloud. On completion of the training over T time-steps, the already employed data (X u ) is permanently removed from the cache to prevent possible breaches of user data in the future. From hereon, we analyze the convergence performance of the proposed asynchronously updating federated learning to address the original optimization problem. The centralized aggregator transmits the model parameters g = w to LNs to initialize and train their respective localized AI models. The UEs, TSCs, and UAVs selectively exchange shallow-only and deep parameters using Algorithm 2 such that the update of each LN i 's local model parameter w i depends on the global model g while the update of the global model g relies on all LNs' local federated learning models. The update of the local model w i depends on the learning algorithm such as Gradient Descent, Stochastic Gradient Descent (SGD), or Randomized Coordinate Descent (RCD). The update of the global model g is obtained as

The update of LN i 's local model w i at time t is expressed as:

where η denotes the learning rate and ▽f (g t , x ik , y ik ) represents the gradient of f (g t , x ik , y ik ) with respect to g t .

, the update of global model g at time t can be given by:

if a resource block is assigned to LN i for transmitting its local model parameters with sufficient transmit-energy. Then based on the work [39] , the federated learning algorithm converges to an optimal global model g * after the learning steps in Algorithm 2. Interested readers may view the proof of the expected convergence rate in [39] .

To synthesize the dataset required for experiments, DJI Matrice 300 RTK drone with URUAV GRAPHENE 6S 22.2V 6000mAh 100C Lipo Battery XT90 [40] having an energy capacity of 133.2Wh has been considered in this work. This battery is commonly used for heavy-duty tasks such as local AI model training. The drone has a flight time of 15 minutes which is considered enough to offload information from mobile UEs not having access to TSCs due to lack of infrastructures or overburdened TSCs. Up to 10 drones are considered in the simulation. In other words, the number of LNs is varied between 2 to 10 in the conducted simulations.

To demonstrate the performance of our envisioned asynchronously updating federated learning model, a public repository of X-ray images with COVID-19, pneumonia, and normal chest X-ray images is employed. Compared to the existing work dealing with small datasets, we prepared a large and robust dataset by merging four different existing datasets consisting of Posteroanterior (PA) chest X-rays based on the coauthors' earlier work in [38] . It is worth noting that the hyperparameter tuning of the underlying ANN and CNN models was done in our earlier work [38] . As for the hyper parameter tuning of our presented federated learning setup, the tuning of deep/shallow parameter exchange ratio/interval is considered for hyperparameter optimization. According to the underlying AI model, two variations of the proposed asynchronously updating federated learning models are simulated based on ANN and CNN, which we refer to as Fed-ANN and Fed-CNN, respectively, for clarity. Furthermore, we define the iteration and time round for evaluating the performance of our proposal. An iteration refers to the number of times the whole process runs (e.g., the local training for the LNs and exchanging parameters). On the other hand, a time round consists of multiple iterations, after which the cloud is updated with the deep weights. In other words, an iteration is an atomic unit relative to the time round. For instance, if the number of iterations is 20 and the number of time rounds is 5, then for the 5th, 10th, 15th, and 20th iteration, the deep model update is triggered in our proposed approach while the shallow model parameter J o u r n a l P r e -p r o o f Journal Pre-proof update takes place during the other iterations. The number of iterations was set to 20, considering the learning loss converged in all considered cases. For all the simulations, the epoch-size and batch-size were set to 50 and 16, respectively. The exponential linear unit (ELU) was employed as the activation function, while Adagrad was used as the optimizer to select and regularly update a preconditioned stochastic gradient descent smartly. For the Fed-ANN model, a fully connected layer followed by the output layer was considered. On the other hand, the Fed-CNN model consists of three convolution layers, followed by three fully connected layers and the output layer. Table 2 Fig. 4a that both models' learning accuracy is over 90% at the fifth time round. Both the variations of our proposal gain higher accuracies with the increasing number of time rounds. However, the Fed-CNN model exhibits superior performance for all the time rounds. This is also illustrated in learning loss, in terms of the mean squared error (MSE), in Fig. 4b .

Next, the impact of iterations on the learning loss performances of Fed-ANN and Fed-CNN is investigated in Fig. 5 . Both Fed-ANN and Fed-CNN demonstrate improvement in the learning with an increasing number of iterations. However, the Fed-CNN model shows a much faster convergence trend, indicating its viability as the candidate deep learning technique for the proposed federated learning framework.

In Fig. 6 , we compare the required execution time and memory requirement for the training phase for the proposed Fed-ANN and Fed-CNN models. While these overheads incurred by Fed-ANN are lower than that of the Fed-CNN model during training, the accuracies in the testing phase between these models as well as the centralized benchmark algorithm [38] are found to be quite similar, as demonstrated in Table 3 . Thus, there is a slight tradeoff between the accuracy and overhead between our proposal's two variations (i.e., Fed-ANN and Fed-CNN), which demonstrates Fed-CNN as the most suitable technique for the COVID-19 detection task.

Finally, Fig. 7 shows how our proposed asynchronous weight update method reduces the network overhead. In this case, 20 rounds are considered, which are enough to demonstrate the overhead reduction performance. Notice that with the larger deep parameter rates, a more overhead reduction is possible by performing more generic parameter exchanges from the local model using the LN to the cloud and only transferring the specific parameters after a relatively high number of time rounds.

In this section, we highlight the open issues and the benefits that the proposed approach has in the B5G context, and then describe the future directions to introduce further security/privacy augmentations that were beyond the scope of the current work. As discussed in Section 2, only few federated learning techniques started to focus on considering the QoS improvement issue along with privacypreserving federated learning that is critical for B5G network users in UAVserved tiny cells where competition for bandwidth will be immense despite the high bandwidth capacity. The proposed work provides a proof-of-concept demonstrating how asynchronously updating weights of the locally trained deep and shallow models can be efficiently scheduled to significantly reduce the overhead reduction while attaining reasonable execution time and memory use as well as achieving high learning accuracy. This is particularly important for emerging health applications for distributed processing and analytics on users' electronic health records, locations, and biomedical image data as demonstrated in the COVID-19 usecase in this paper. The opportunistic use of the temporal synchronization in a smart manner, thus, enlarges the scale of training data with resource-constrained and highly mobile B5G mobile under dynamically changing RAT conditions, and at the same time, protects the privacy of the user data.

As an open research issue, we provide a theoretical argumentation, in a high-level manner, describing possible integration of the Blockchain technol-J o u r n a l P r e -p r o o f Journal Pre-proof ogy and/or methods to enable Blockchain-based applications with the proposed asynchronously updating federated learning framework. This is due to the fact that the security of local parameters, the learning quality, and the varying computing and communication resources, remain unexplored topics in the existing federated learning schemes [41] including our proposal. Blockchain is an advanced technology which has modified the ways to store, exchange, and record the data using a public ledger so that the transactions are immutable [42, 43, 44] . On the other hand, the aim of federated learning is somewhat limited to applications orchestrated by the centralized aggregator as demonstrated in Algorithm 1. Hence, the robustness of the centralized aggregator relies upon the secrecy performance and resiliency of the central server. In other words, the aggregator may act as a single-point-of-failure by jeopardizing the entire federated learning framework [45] . Furthermore, despite the local data from LNs not being shared explicitly in their original format, knowledgeable adversaries may approximate the raw data reconstruction during the aggregation process [45] . Thus, privacy leakage may occur during model aggregating by outsider threats [46] . Also, concealing the COVID-19 training data may present the malicious users with an opportunity to inject backdoors into the trained model [45, 47] . Researchers are recently considering adversarial models whereby an attacker may be able to inject backdoors to the trained model during federated learning, and then can leverage the backdoor to make the model misclassify at a later time. In the future, how more seamless integration between asynchronous federated learning and blockchain should be integrated needs to be carefully investigated to avoid the aforementioned open research issues. Furthermore, this work can be considered as proof-of-concept and can be extended toward other pandemic monitoring given that the dataset is available from public health agencies and other relevant organizations at the national level.

Next, because of the heterogeneous resources and capabilities of different RATs and UEs in B5G networks, the selection of participating nodes (e.g., a mobile UE as opposed to a more powerful UAV node or a even more powerful terrestrial gNB) may have a dominating impact on the performance and convergence-time of federated learning. Furthermore, in the global model aggregator/server-side, initialization level of the global model and fine-tuning parameters such as the duration for each iteration, as well as duration for local model exchange for deep/shallow parameters, need to be taken into account in the future. 23 

In this paper, we highlighted the importance of formulating an intelligent pandemic response network, particularly during the ongoing pandemic of COVID-19. We presented a drone-based heterogeneous computing system model to support the patient/user entities or user equipment (UEs), which have the capability of collecting their health data but are not bound to sharing the collected data with the drones/other base stations to be delivered to the central cloud managed by the public health authority and/or other hospitals. Instead, they share the wisdom with the network entities in more frequent, small-sized locally trained shallow model parameters and infrequent, heavy deep, or specific model parameters regarding their monitored health data. On the other hand, the UAVs poll each UE systematically, exchange them further with other UAVs, and forward the models to the cloud. They also help, in addition to terrestrial base stations, to deliver the updated global model to the UEs. Using this smart technique, the user data's privacy is inherently respected, and the network overhead is significantly reduced. Through the performance evaluation on a particular use-case using X-ray images containing COVID-19, pneumonia, and normal samples, the effectiveness of two variants of our proposed asynchronously updating federated learning approach demonstrated the effectiveness as a viable pandemic response network methodology.

Smart IoT solutions for combating the COVID-19 pandemic

UV-CDS: An energy-efficient scheduling of UAVs for premises sterilization

COVID watch | pandemic response network

COVID-19 pandemic response network -full text view

On a novel adaptive UAV-mounted cloudlet-aided recommendation system for LBSNs

On COVID-19 prediction using asynchronous federated learning-based agile radiograph screening booths

Towards federated learning at scale: System design

Cyber security meets artificial intelligence: a survey

A survey on deep learning techniques for privacy-preserving

A review of privacy-preserving machine learning classification

A survey on collaborative deep learning and privacy-preserving

Threats to federated learning: A survey

Federated learning in mobile edge networks: A comprehensive survey

A survey on federated learning systems: Vision, hype and reality for data privacy and protection

Federated learning: Challenges, methods, and future directions

Preserving data privacy via federated learning: Challenges and solutions

Secure, privacypreserving and federated machine learning in medical imaging

Federated machine learning: Concept and applications

Federated learning for wireless communications: Motivation, opportunities, and challenges

Federated learning

A comprehensive survey of privacy-preserving federated learning: A taxonomy, review, and future directions

Hcp: Heterogeneous computing platform for federated learning based collaborative content caching towards 6g networks

Communication-efficient federated deep learning with asynchronous model update and temporally weighted aggregation

The future of digital health with federated learning

Dynamic fusion based federated learning for COVID-19 detection

Experiments of federated learning for COVID-19 chest X-ray images

Triaging COVID-19 patients: 20 hospitals in 20 days build AI model that predicts oxygen needs

A pandemic AI engine without borders

HCP: Heterogeneous computing platform for federated learning based collaborative content caching towards 6G networks

Routing or computing? the paradigm shift towards intelligent computer network packet transmission based on deep learning

Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach

A UAV-based moving 5G RAN for massive connectivity of mobile users and IoT devices

Towards power-efficient aerial communications via dynamic multi-UAV cooperation

Using multi-layer gated recurrent units to identify electron transport proteins

Convolutional gated recurrent neural network incorporating spatial features for audio tagging

Communication-efficient federated deep learning with asynchronous model update and temporally weighted aggregation

Deep learning-based chest radiograph classification for COVID-19 detection: A novel approach

A joint learning and communications framework for federated learning over wireless networks

2V 6000mAh 100C 6S Lipo Battery XT90 Plug for RC Racing Drone

Blockchain and federated learning for 5g beyond

On securing electronic healthcare records using hyperledger fabric across the network edge

Smart parking system with privacy preservation and reputation management using blockchain

Verifiable badging system for scientific data reproducibility

When federated learning meets blockchain: A new distributed learning paradigm

Federated learning: Challenges, methods, and future directions

Blockfla: Accountable federated learning via hybrid blockchain architecture, CODASPY '21

Authors would like to thank Alfaisal University for funding and supporting this work through the Internal Research Grant IRG21207.

Please check the following as appropriate: All authors have participated in (a) conception and design, or analysis and interpretation of the data; (b) drafting the article or revising it critically for important intellectual content; and (c) approval of the final version. This manuscript has not been submitted to, nor is under review at, another journal or other publishing venue. The authors have no affiliation with any organization with a direct or indirect financial interest in the subject matter discussed in the manuscript o The following authors have affiliations with organizations with direct or indirect financial interest in the subject matter discussed in the manuscript:Author's name Affiliation