key: cord-295116-eo887olu authors: Chimmula, Vinay Kumar Reddy; Zhang, Lei title: Time Series Forecasting of COVID-19 transmission in Canada Using LSTM Networks() date: 2020-05-08 journal: Chaos Solitons Fractals DOI: 10.1016/j.chaos.2020.109864 sha: doc_id: 295116 cord_uid: eo887olu On March 11(th) 2020, World Health Organization (WHO) declared the 2019 novel corona virus as global pandemic. Corona virus, also known as COVID-19 was first originated in Wuhan, Hubei province in China around December 2019 and spread out all over the world within few weeks. Based on the public datasets provided by John Hopkins university and Canadian health authority, we have developed a forecasting model of COVID-19 outbreak in Canada using state-of-the-art Deep Learning (DL) models. In this novel research, we evaluated the key features to predict the trends and possible stopping time of the current COVID-19 outbreak in Canada and around the world. In this paper we presented the Long short-term memory (LSTM) networks, a deep learning approach to forecast the future COVID-19 cases. Based on the results of our Long short-term memory (LSTM) network, we predicted the possible ending point of this outbreak will be around June 2020. In addition to that, we compared transmission rates of Canada with Italy and USA. Here we also presented the 2, 4, 6, 8, 10, 12 and 14(th) day predictions for 2 successive days. Our forecasts in this paper is based on the available data until March 31, 2020. To the best of our knowledge, this of the few studies to use LSTM networks to forecast the infectious diseases. Every infectious disease outbreak exhibits certain patterns and such patterns needed to be identified based on transmission dynamics of such outbreaks. Intervening measures to eradicate such infectious diseases rely on the methods used to evaluate the outbreak when it occurs. Any outbreak in a country or province usually occurs at different levels of magnitude with respect to time i.e. seasonal changes, adaptation of virus over time. Usually patterns exhibited in such scenarios are non-linear in nature and this motivates us to design the system that can capture such non-linear dynamic changes. With the help of these non-linear systems, we can describe the transmission of such infectious diseases. In [1] [2] a transmission model for malaria and in [3] a mathematical model for analysing dynamics of tuberculosis has been developed to study the transmission using mathematical models. In [4] a laplacian based decomposition is used to solve the non-linear parameters in a Pine Witt disease. A modified SIRS model in [5] successfully helped to control the syncytial virus in infants. Similarly mathematical models presented in [6] , [7] helped clinicians to better understand the characteristics of human liver and transmission of dengue outbreak. Most of the Data driven approaches used in previous studies [8] are linear methods and often neglects the temporal components in the data. They depend upon regression without non-linear functions and failed to capture the Regressive (AR) methods overwhelmingly depends on assumptions and such models are difficult for forecasting realtime transmission rates. Wide range of statistical and mathematical models [9] [10] have been proposed to model the transmission dynamics of current COVID-19 epidemic. In many cases, these models are not able to fit the given data perfectly and accuracy is also low while predicting the growth of COVID-19 transmission. R0 is a popular statistical method specifically used to model an infectious disease. Often referred as âĂŸreproduction numberâĂŹ because, the infections reproduce itself with respect to time. R0 forecasts the number of people can get the infection from the infected person. In this model, an extra weight is applied to the person who never infected the current disease nor vaccinated. If the value of R0 of a disease is 10, then the infected person will spread the disease to 10 other people surrounding him. In [11] authors used R0 method to find the infection rate of novel virus on diamond princes cruise ship [11] . However, in such method it is difficult to find the starting point of the infectious disease by identifying patient zero and the people he interacted with during his incubation period. It is worth noting that mathematical models presented in [12] , [13] , [14] can be used to solve the complex non-linear patterns of infectious diseases. Even though these epidemiological models are good at capturing vital components of an infectious disease, parameters of these models required several assumptions. Such hypothesized parameters would not fit the data perfectly and precision of such models will be low. Meanwhile, in engineering applications [15] , model parameters are calculated with the help of real-time data. Similar approach was used in this research to find the model parameters instead of assumptions. In order to overcome the barriers of statistical approaches, we developed the Deep Learning based network to predict the real-time transmission. Our model could help public health care providers, policy makers to make necessary arrangements to tackle the rush of potential COVID-19 patients. This experiment is based on the data sets of confirmed COVID-19 cases available until March 31, 2020. Artificial Intelligence and mobile computing are one of the key factors for the success of technology in health care systems [16] . In the world of smart devices, data is being generated in the unprecedented way than ever before and promoted the role of machine learning in healthcare [16] . The world today is more connected than ever before this helped to share the real time infectious data between the countries. The distinctive feature of artificial intelligence is its flexibility, domain adaptation and economical to integrate with existing systems. Over the last few weeks, many researchers came up with several mathematical models to predict the transmission of novel corona virus [17] [18] . The major drawbacks of the existing models are linear, non-temporal and several assumptions while modelling the network. First of all, the covid-19 is a time series data set and it is highly recommended to use the sequential networks to extract the patterns from it. Second of all, the data we are dealing with is dynamic in nature so by using statistical and epidemiological models, results are often vague [19] [20] . In [21] , [22] , [23] , [24] researchers used deep learning based LSTM networks to forecast COVID-19 infections. The LSTM models used in the above networks could not able to represent the spatio-temporal components simultaneously. In this paper we addressed the above problem by modifying the internal connections. In our modified LSTM cells, We have established the alternative connections between the input and output cells. This type of connections not only helps the networks to preserve spatio-temporal components, but also to transfer the historical information to the next units. In this paper, we made an effort to predict the outbreak of COVID-19 based on past transmission data. First of all, coherence of input data needs to be analyzed in order to find the key feature i.e. number of new cases reported with respect to the previous day infections. After selecting the key parameters of the network, several experiments was conducted to find the optimal model that can predict future infections with minimum error. Previous studies on COVID-19 predictions, did not considered the recovery rate while developing the model. In this research, we considered the recovery rate as one of the features while building our model. From the design point of view, when a crisis occurs, algorithms tend to assign high probability and completely neglects the previous information which leads to biased predictions. We addressed this issue in our literature and solved this by using LSTM networks. Our results are expected to alert the public health care providers of Canada to prepare themselves for the crisis against COVID-19. With the help of this real-time forecasting tool, front-line clinical staff will be alerted before the crisis. The rest of this paper is structured as follows: section II describes methods, datasets and LSTM models used in this paper. In Section III, we have discussed our findings and in Section IV, concussion and future work was discussed The COVID-19 data used in this research is collected from Johns Hopkins University and Canadian Health authority, provided with number of confirmed cases until March 31, 2020. The data set also includes number of fatalities and recovered patients by the end of each day. The dataset is available in the time series format with date, month and year so that the temporal components are not neglected. A wavelet transformation [25] is applied to preserve the timefrequency components and it also mitigates the random noise in the dataset. The fundamental point to represent and forecast the trends of current is to select conventional functions to fit the data. The COVID-19 dataset is divided into training set (80%) on which our models are trained and testing set (20%) to test the performance of the model. A large part of real-world datasets are temporal in nature. Due to its distinctive properties, there are numerous unsolved problems with wide range of applications. Data collected over regular intervals of time is called time-series (TS) data and each data point is equally spaced over time. TS prediction is the method of forecasting upcoming trends/patterns of the given historical dataset with temporal features. In order to forecast COVID-19 transmission, it would be effective if input data has temporal components and it is different from traditional regression approaches. A time series (TS) data can be break downed into trend, seasonality and error. A trend in TS can be observed when a certain pattern repeats on regular intervals of time due to external factors like lockdown of country, mandatory social distancing, quarantines etc. In many real-world scenarios, either of trend or seasonality are absent. After finding the nature of TS, various forecasting methods have to be applied on given TS Given the TS, it is broadly classified into 2 categories i.e. stationary and non-stationary. A series is said to be stationary, if it does not depend on the time components like trend, seasonality effects. Mean and variances of such series are constant with respect to time. Stationary TS is easier to analyze and results skilful forecasting. A TS data is said to nonstationary if it has trend, seasonality effects in it and changes with respect to time. Statistical properties like mean, variance, sand standard deviation also changes with respect to time. In order to check the nature (stationarity and non-stationarity) of the given COVID-19 dataset, we have performed Augmented Dickey Fuller (ADF) test [26] on the input data. ADF is the standard unit root test to find the impact of trends on the data and its results are interpreted by observing p-values of the test. If P is between 5-1%, it rejects the null hypothesis i.e. it does not have a unit root and it is called stationary series. If P is greater than 5% or 0.05 the input data has unit root so it is regarded as non-stationary series. Before diving into the model architecture, it is crucial to explain the internal mechanisms of LSTM networks and reasons behind using it instead of traditional Recurrent Neural Networks. Recurrent LSTM networks has capability to address the limitations of traditional time series forecasting techniques by adapting nonlinearities of given COVID-19 dataset and can result state of the art results on temporal data. Each block of LSTM operates at different time step and passes its output to next block until the final LSTM block generates the sequential output. As of this writing, RNNs with blocks (LSTM) are the efficient algorithms to build a time series sequential model. The fundamental component of LSTM networks is memory blocks, which was invented to tackle vanishing gradients by memorizing network parameters for long durations. Memory block in LSTM architecture are similar to the differential storage systems of a digital systems. Gates in LSTM helps in processing the information with the help of activation function (sigmoid) and output is in between 0 or 1. Reason behind using sigmoid activation function is because, we need to pass only positive values to the next gates for getting a clear output. The 3 gates of LSTM network are represented with the following equations below: Where: = function of input gate = function of forget gate = function of output gate = coefficients of neurons at gate (x) −1 = result from previous time step = input to the current function at time-step t = bias of neurons at gate (x) Input gate in the first equation gives the information that needs to be stored in the cell state. Second equation throws the information based on the forget gate activation output. The third equation for output gate combines the information from the cell state and the output of forget gate at time step âĂŸtâĂŹ for generating the output. The internal block diagram of LSTM block used in this study is shown in 1 The motivation behind initiating self-loops is to create a path so that gradients or weights can be shared for long durations. Especially, this is useful while modelling deep networks where vanishing gradient is a frequent issue to deal with. By adjusting weights as self-looped gates, we can adjust the time scale to detect the dynamically changing pa- rameters. Using the above techniques, LSTMs are able to produce the state-of-the-art results in [27] . The network architecture used in this study is shown in 2 The methods used in this study are based on data guided approaches and are completely different from previous studies. Our approaches and predictive outcomes will provide assistance for restricting the infections and possible elimination of current COVID-19 pandemic. We trained our network with data until March 31, 2020 reported by Canadian health authority. In this study we found that policies or decisions taken by government will greatly affect the current outbreak.Several studies on forecasting of COID-19 transmission are based on the R0 method however, they didn't include the sensitivity analysis to find the important features. We examined our model predictions using mean square error (MSE). In figure 4 we plotted the total number of confirmed cases and forecasted COVID-19 cases in Canada as a function of time. From the figure we can observe that, Canada didnâĂŹt witness its peak yet and it is expected number of cases will soon increase exponentially despite the social distancing. Although our model achieved better performance when compared with other forecasting models, it is unfortunate that transmissions are following increasing trend. The rate of infections in USA, Italy and Spain are growing exponentially meanwhile, the number of infections in Canada are increasing linearly in figure 3. If Canadians follow the regulations strictly, the number of confirmed cases will soon decline. In our LSTM model-1 we trained and tested our network on Canadian dataset; the RMSE error is 34.83 with an accuracy of 93.4% for short term predictions in Canada. Meanwhile, based on our testing/validation dataset the RMSE error is about 45.70 with an accuracy of 92.67% for long term predictions. The predictions of LSTM model are shown in 4 with solid red line. It shows that our model was able to capture the dynamics of the transmission with minimum loss. From the figure 4 we can say that Canada witnessed linear growth in cases until March 16 2020 after its first confirmed case. The current epidemic in Canada is predicted to continue until June 2020. Our second LSTM model-2 is trained on Italian dataset to predict short-term and long-term infections in Canada. For short term predictions, the RMSE error is about 51.46 which is higher than previous model. Accord-COVID-19 forecasting using LSTM Networks ing to this second model within 10 days, Canada is expected to see exponential growth of confirmed cases. It was a challenging task to forecast the dynamics of transmission based on small dataset. Even though COVID-19 outbreak started in Canada around early January, the consistent epidemiological data wasn't released until early February. Because of small dataset several statistical models struggled to select the optimal parameters and several unknown variables led to uncertainty in their predictions. LSTM model is different from statistical methods in many ways for instance, the proposed LSTM network fits the real-time data and without any assumptions while selecting hyperparameters. It was able to overcome the parameter assumptions using cross validation and achieved better performance by reducing the uncertainty. After reaching the inflection point, the recovery rate will start decrease rapidly and death rate may increase at the same time as shown in figure 5 . In order to find the trend of the infections we decomposed the given series and the trend of infections is increasing with respect to time. Further, number of infections followed increasing trend from Sunday to Tuesday and followed decreasing trend until Saturday as shown in figure 6 . As we are still under the stage of dilemma about the current situation of COVID-19 because, the accuracy of our estimates is bounded with a lot of external factors. So, it is recommended to conduct the follow-up study after this experiment to be more precise about the dynamics of this novel infectious disease. The actual number of cases might be higher than the cases reported by the government because, of the backlog of test results and some people will be immune before even testing. All the above factors may lead to discrepancy of our model estimations. Even though we addressed data imbalance by using statistical methods like interpolation and re-sampling yet we couldnâĂŹt represent patients who are on incubation period or not tested. Other problem while modelling current pandemic is that, people COVID-19 forecasting using LSTM Networks Figure 4 : Predictions of the LSTM model on current exposed and infectious cases (Red solid line). The red dotted lines represents the sudden changes from where number of infections started following exponential trend. The black dotted lines in the figure represents the training data or available confirmed cases travelling between the provinces. Based on our sensitivity analysis our projections may go down if current trials on potential vaccines achieves fruitful results. Finally, in order to minimize the bias on our training algorithm we introduced regularization. Further, by training our network inversely, we found that outbreak in Canada started around early January but, it was not reported until January last week. Even without the knowledge of 1st case, our inverse training will help governments to better understand the outbreak of COVID-19 and helps then to prevent such outbreaks in future. The patterns from the data reveals that prompt and effective approaches taken by Canadian public health authorities to minimize the human exposure is showing a positive impact when compared with other countries like USA and Italy 3. Rate of transmission in Canada is following linear trend while in USA is witnessing an exponential growth of transmissions. However, it is too early to draw the conclusions about the current epidemic. After simulations and data fitting, our model predicted Canada would reach peak within 2 weeks from now. However, the current outbreak resembles early 20th century Spanish flu [28] , which killed millions of people and lasted for 2 COVID-19 forecasting using LSTM Networks years. Based on our model simulations, the current COVID-19 pandemic is expected to end within 3 months from now. Due to some unreported cases, a small number infection clusters may appear until December 2020. However, recent technological improvements and international cooperation between countries may even reduce the duration current pandemic. To sum up, this is the first study to model the infections disease transmission model to predict the gravity of COVID-19 in Canada using deep learning approaches. Based on our current findings, provinces that have implemented social distancing guidelines before the pandemic has less confirmed cases than other provinces 3. For instance, Saskatchewan issued social distancing guidelines 2 weeks ahead than Quebec which has half of the confirmed cases in Canada. Our results could help Canadian government to monitor the current situation and use our forecasts to prevent further transmissions. We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property. We further confirm that any aspect of the work covered in this manuscript that has involved human patients has been conducted with the ethical approval of all relevant bodies and that such approvals are acknowledged within the manuscript. IRB approval was obtained (required for studies and series of 3 or more cases) Written consent to publish potentially identifying information, such as details or the case and photographs, was obtained from the patient(s) or their legal guardian(s). The International Committee of Medical Journal Editors (ICMJE) recommends that authorship be based on the following four criteria: 1. Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND 2. Drafting the work or revising it critically for important intellectual content; AND 3. Final approval of the version to be published; AND 4. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All those designated as authors should meet all four criteria for authorship, and all who meet the four criteria should be identified as authors. For more information on authorship, please see http://www.icmje.org/recommendations/browse/roles-andresponsibilities/defining-the-role-of-authors-and-contributors.html#two. All listed authors meet the ICMJE criteria. We attest that all authors contributed significantly to the creation of this manuscript, each having fulfilled criteria as established by the ICMJE. One or more listed authors do(es) not meet the ICMJE criteria. We believe these individuals should be listed as authors because: We confirm that the manuscript has been read and approved by all named authors. We confirm that the order of authors listed in the manuscript has been approved by all named authors. The Corresponding Author declared on the title page of the manuscript is: This author submitted this manuscript using his/her account in editorial submission system. We understand that this Corresponding Author is the sole contact for the Editorial process (including the editorial submission system and direct communications with the office). He/she is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that the email address shown below is accessible by the Corresponding Author, is the address to which Corresponding Author's editorial submission system account is linked, and has been configured to accept email from the editorial office of International Journal of Women's Dermatology: Someone other than the Corresponding Author declared above submitted this manuscript from his/her account in editorial submission system: We understand that this author is the sole contact for the Editorial process Malaria transmission dynamics of the anopheles mosquito in kumasi, ghana Bifurcation analysis of a mathematical model for malaria transmission Mathematical analysis of the transmission dynamics of hiv/tb coinfection in the presence of treatment Semianalytical study of pine wilt disease model with convex rate under caputo-febrizio fractional order derivative A new fractional hrsv model and its optimal control: A non-singular operator approach A new study on the mathematical modelling of human liver with caputofabrizio fractional derivative A new fractional modelling and control strategy for the outbreak of dengue fever Bridging the gap between evidence and policy for infectious diseases: How models can aid public health decision-making Application of the arima model on the covid-2019 epidemic dataset Forecasting of covid-19 confirmed cases in different countries with arima models Estimation of the reproductive number of novel coronavirus (covid-19) and the probable outbreak size on the diamond princess cruise ship: A datadriven analysis The fractional features of a harmonic oscillator with position-dependent mass New aspects of time fractional optimal control problems within operators with nonsingular kernel A new feature of the fractional euler-lagrange equations for a coupled oscillator using a nonsingular operator approach Deep learning for real-time gravitational wave detection and parameter estimation: Results with advanced ligo data COVID-19 forecasting using LSTM Networks The âĂIJinconvenient truthâĂİ about ai in healthcare Preliminary estimation of the basic reproduction number of novel coronavirus (2019-ncov) in china, from 2019 to 2020: A data-driven analysis in the early phase of the outbreak Transmission potential and severity of covid-19 in south korea Updating of covariates and choice of time origin in survival analysis: problems with vaguely defined disease states Strong consistency of least-squares estimation in linear regression models with vague concepts Machine learning approach for confirmation of covid-19 cases: Positive, negative, death and release Multiple-input deep convolutional neural network model for covid-19 forecasting in china Prediction for the spread of covid-19 in india and effectiveness of preventive measures Neural network based country wise risk prediction of covid-19 Wavelet transform domain filters: a spatially selective noise filtration technique Lag order and critical values of the augmented dickey-fuller test Insights into lstm fully convolutional networks for time series classification A pandemic warning? No funding was received for this work. Vinay Kumar Reddy Chimmula: Conceptualization of this study, Methodology, Software, Writing -Original draft preparation,Critical revision of the manuscript for important intellectual content. Lei Zhang: Data curation,Critical revision of the manuscript for important intellectual content,Supervision and material support, regular feedback after each update. Manuscript title: The authors whose names are listed immediately below certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers' bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript. The authors whose names are listed immediately below report the following details of affiliation or involvement in an organization or entity with a financial or non-financial interest in the subject matter or materials discussed in this manuscript. Please specify the nature of the confl ict on a separate sheet of paper if the space below is inadequate. We wish to draw the attention of the Editor to the following facts, which may be considered as potential conflicts of interest, and to significant financial contributions to this work: The nature of potential conflict of interest is described below:No conflict of interest exists.We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome. Funding was received for this work.All of the sources of funding for the work described in this publication are acknowledged below:(including editorial submission system and direct communications with the office). He/she is responsible for communicating with the other authors, including the Corresponding Author, about progress, submissions of revisions and final approval of proofs.We the undersigned agree with all of the above.Author's name (Fist, Last) Signature Date All persons who meet authorship criteria are listed as authors, and all authors certify that they have participated sufficiently in the work to take public responsibility for the content, including participation in the concept, design, analysis, writing, or revision of the manuscript. Furthermore, each author certifies that this material or similar material has not been and will not be submitted to or published in any other publication before its appearance in the Hong Kong Journal of Occupational Therapy. Please indicate the specific contributions made by each author (list the authors' initials followed by their surnames, e.g., Y.L. Cheung). The name of each author must appear at least once in each of the three categories below. Conception and design of study: VKR Chimmula acquisition of data: VKR Chimmula analysis and/or interpretation of data: VKR Chimmula; L Zhang Drafting the manuscript: VKR Chimmula, revising the manuscript critically for important intellectual content: L Zhang; VKR Chimmula Approval of the version of the manuscript to be published (the names of all authors must be listed):VKR Chimmula, L Zhang.