key: cord-0981166-jum025b7
authors: Ngaffo, Armielle Noulapeu; Ayeb, Walid El; Choukair, Zièd
title: Service recommendation driven by a matrix factorization model and time series forecasting
date: 2021-05-16
journal: Appl Intell
DOI: 10.1007/s10489-021-02478-0
sha: 97a65244602a0e6d86c74f4588bf0c5d64e22129
doc_id: 981166
cord_uid: jum025b7

The rise of high-quality cloud services has made service recommendation a crucial research question. Quality of Service (QoS) is widely adopted to characterize the performance of services invoked by users. For this purpose, the QoS prediction of services constitutes a decisive tool to allow end-users to optimally choose high-quality cloud services aligned with their needs. The fact is that users only consume a few of the broad range of existing services. Thereby, perform a high-accurate service recommendation becomes a challenging task. To tackle the aforementioned challenges, we propose a data sparsity resilient service recommendation approach that aims to predict relevant services in a sustainable manner for end-users. Indeed, our method performs both a QoS prediction of the current time interval using a flexible matrix factorization technique and a QoS prediction of the future time interval using a time series forecasting method based on an AutoRegressive Integrated Moving Average (ARIMA) model. The service recommendation in our approach is based on a couple of criteria ensuring in a lasting way, the appropriateness of the services returned to the active user. The experiments are conducted on a real-world dataset and demonstrate the effectiveness of our method compared to the competing recommendation methods.

The rapid development of broadband access technologies has led to an escalation of rich and varied cloud services and applications. The plethora of existing services complicates decision-making for internet users. Recommender systems address this issue by proposing algorithms to predict users' sensitivity for services aligned with their needs. According to the literature review, recommender systems are popularly implemented through Collaborative Filtering (CF) [22] . Collaborative-filtering-based recommender systems are mainly organized into two types, namely memory-based CFs and model-based CFs. Among memory-based CFs, we identify user-based CFs [11, 12] and item-based CFs [14] whose respective approaches are based on the study of interactions between users and items to assess interuser similarities and inter-item similarities. Memory-based CFs are based on the assumption that, within a group of users with similar or identical behaviors, the previous experiences of certain users with regard to consumed services, can help to predict those of others relative to the same services. Memory-based CFs are easy to implement and present an acceptable quality of prediction; however, they are based on resource-intensive algorithms due to the fact that computational calculations are applied to the full data matrix [24] . In other words, memory-based CFs display scalability limitations, an inappropriateness for large datasets, poor performances in case of data sparseness and during the cold start problem for new users and new items [2] . Model-based CFs overcome the limitations displayed by memory-based CFs. Indeed, model-based CFs present the advantage of their effectiveness in the case of large datasets and data sparsity. Among model-based CFs, we identify those based on matrix factorization techniques, those based on clustering and others based on mathematical models such as Bayesian estimate, deep learning, random walker, etc [30] .

The static nature of the data is an assumption on which classic recommender systems are modeled. However, a realistic recommendation approach should take into account the variability of users' behavior over time [4] . To address this issue, time-aware recommenders systems have been designed with the objective of integrating the dynamicity of users' needs and behaviors over time [21] . Existing timeaware recommendation solutions only predict the current moment and display poor performances in the case of data sparsity. In reality, faced with a multitude of services, users invoke and co-invoke only a small group of services, thus causing increased porosity of the data matrix with a large quantity of missing data. The aforementioned inadequacies of the state-of-the-art recommendation methods constitute our motivation to propose a data sparsity resilient and high-accurate recommendation approach that aims to predict the current needs but also the near future needs of end-users. We propose a service recommendation approach based on a latent feature unsupervised learning coupled with a time series forecasting method. In this paper, our contribution is highlighted through the following aspects:

-By applying a flexible matrix factorization technique, our method addresses the data sparsity problem in the presence of a large amount of missing data. -Our approach is based on time series forecasting method for the prediction of the future time slot t c+1 using the prediction of the current time slot t c . The prediction of the future time slot anticipates the near future needs of the active user. -The variability of users' behavior and the impact of data recentness are integrated into our approach by the definition of a decay function allowing to sketch the evolution of user interest profiles over time. -The services recommended to the active user meet a couple of criteria (q c 0 , q c 0 +1 ); thus ensuring the exclusive selection of services whose prediction of the current time t c , and that of the future time slot t c+1 are among the highest. -Extensive experiments are conducted on real-world datasets and highlight the effectiveness of our proposal compared to state-of-the-art recommendation methods.

The remainder of this paper is structured as follows: Section 2 presents state-of-the-art recommendation methods, Section 3 presents our recommendation approach. Section 4 describes the experiments and the results obtained. Section 5 concludes this paper and presents our perspectives.

The growth of the number of cloud services and the difficulty for internet users to choose services matching their needs have motivated scientific research in the field of recommender systems. In this section, we survey the state-of-the-art methods related to recommender systems.

Memory-based CFs present the advantage of their explainability and easy-implementation. They are based on the assumption that within a group of users with identical or similar behaviors, the previous experiences of certain users can be used to predict those of others with regard to the services never invoked. Within memory-based CFs, a distinction is made between user-based CFs, item-based CFs and user-item-based CFs [30] . A cloud service recommendation based on user-based CF is proposed by the authors [10, 25] . The authors assess inter-user similarities using the Spearman correlation coefficient [6] . While the authors [32] propose a Cloud service recommendation method based on the calculation of inter-user similarities using the Kendall rank correlation coefficient [20] . The researchers [14] offer an efficient privacy-preserving recommendation method based on an item-based CF. The authors assess the inter-item similarities using a modified Pearson measure to preserve user privacy. In [31] , the authors offer a web service recommendation approach based on hybrid user-item-based CFs. Using Pearson correlation measure [12] , the authors assess inter-user similarities and inter-service similarities to reconcile the advantages of user-based CFs and item-based CFs.

The aforementioned recommendation methods are greedy in terms of computational resources, non-scalable and low-accurate. Indeed, similarity computations on large datasets are hard-achievable. Furthermore, these memorybased recommendation approaches display poor performances in the case of data sparseness; meaning in the presence of a large amount of missing data. To overcome these shortcomings, model-based recommender systems are widely adopted for their effectiveness.

Model-based CFs overcome the limitations displayed by memory-based CFs. Indeed, model-based CFs are highly accurate, scalable and suitable for large datasets [2] . In the literature review, we distinguish the model-based CF implemented based on the matrix factorization technique, those whose implementation is based on clustering-based algorithms and others based on deep learning, the bayesian network, etc. Yang et al [30] . The authors [16] offer a recommendation approach based on both user-based CFs and item-based CFs. The method proposed by the authors is based on the Bayesian estimate of the probability with which the active user rates an item. The authors' method reconciles the efficiency of model-based CFs and the easy-understandability of memory-based CFs. Researchers [33] offer a cold start recommendation method. To alleviate the cold start problem for new elements in the recommender system, the authors adopt the matrix factorization technique by reducing the original high-rank matrix to low-rank matrices. In [8] , the authors predict users' tastes by applying matrix factorization based on a Bayesian probabilistic model. The authors' proposal in [13] performs an improved matrix factorization technique for predicting metabolite-disease associations. In [15, 23] , the service recommendation is based on a matrix factorization supported by deep learning algorithms. The model-based CFs are scalable, robust and efficient even in case of data sparsity, case of a cold start problem for newly added users and items; they are suitable for large datasets. However, the above-mentioned methods do not take into account the dynamic nature of users' needs over time.

Classic recommender systems assume static users' behavior over time. In reality, user preferences display variability over time which is ignored by existing recommenders systems. To address this issue, time-aware recommender systems [17, 18, 29] have been developed to integrate the dynamic nature of the service performances. In [28] , the authors offer a time-aware service recommendation approach based on the tensor factorization technique. The authors perform a personalized QoS prediction based on a latent feature learning. However, the method proposed by the authors does not integrate the variability of users' needs over time. In other words, the data recentness is ignored by the authors' approach; all data whatever their seniority, contribute identically in the prediction process. Following [5] , they propose a cloud service recommendation approach based on user-based CF and an ARIMA model. Their method focused on the inter-user similarity evaluation displays limitations faced with the cold start problem and data sparsity. The authors in [7] propose a forecasting ARIMA-based model to predict the Coronavirus evolution throughout the world. Of course, in the literature, some time-aware recommendation approaches to consider the users' variability over time, use a decay function. Most often, the decay function is coupled to memory-based approaches [5, 19, 26] . The matrix factorization technique is popularly adopted in several time-aware recommender systems thanks to its efficiency. However, a properly chosen decay function could help to accurately model the dynamic character of users' tastes over time. In addition, the baseline matrix factorization technique used in the state-of-the-art time-aware recommender systems can be also improved to accurately model intricate user-items interactions.

The novelty of our proposal relies on a fitting proposed decay function embedded into a flexible matrix factorization process. The proposed decay function properly sketches the variability of users' needs over time and therefore, contributes to the refinement of users' tastes prediction. Moreover, the used matrix factorization technique is doubly biased to accurately model intricate user-service interactions. In addition, ARIMA models are most often used for memory-based recommender systems that are very poor-scalable. To remedy this limitation, our proposal uses the effectiveness of ARIMA model combined with the robustness of model-based recommender systems such as matrix factorization-based ones.

In this paper, we address the data sparsity problem and the problem of service targeting. We are interested in the proposal of a service recommendation model which is data sparseness robust and which considers the dynamic nature of needs and the behavior of users over time. The approach presented in this article aims to predict the current moment and also the future time slot in order to anticipate the near future users' needs. Our recommendation method is based on two main prediction phases. The first step is to predict the current time interval. At this phase, a flexible matrix factorization technique is applied to a set of instantaneous data matrices in order to perform the QoS prediction for the current time slot. The second step is devoted to predicting future time intervals using a time series forecasting method based on an AutoRegressive Integrated Moving Average (ARIMA) model. The next section presents our TASERM approach.

Our approach aims to make a QoS prediction of the current time interval and the future time slot based on past historical QoS experiences. The purpose is to return to the active user, services which meet his current and near future needs. In this way, the relevance of the results returned to the enduser is extended over time in order to durably meet the current and future expectations of the active user. Figure 1 represents our TASERM's model workflow. It works as follows:

-The active user invokes cloud services thus generating QoS data which are stored in the database. These QoS data are derived from service invocations spread over regular time slots. -During the data preprocessing, QoS data are normalized and a logistic function is applied on collected data to consider the dynamic character of users' behaviors over time. -Thereafter, for each time slot, a user-service matrix is constructed. Therefore, the |Ψ | instantaneous userservice matrices are obtained knowing that |Ψ | represents the number of time slots in the observation window. -The QoS prediction is carried out by a flexible matrix factorization technique in order to predict the current time slot t c . -The future time slot t c+1 is predicted using a time series forecasting method based on an ARIMA model. -Services whose predicted QoS data at the current time t c and future time slot t c+1 are the highest, are recommended to the active user.

The next subsection presents the mathematical formulation of our problem.

We consider a recommender system in which m users belonging to the set Fig. 2 ). At each invocation of a service s by a user u during a time slot t k , the observed QoS value is recorded as q s u (t k ) in the instantaneous user-service matrix M k = q s u (t k ) m×n where u ∈ U , s ∈ S, t k ∈ Ψ . Deductively, on the observation window Ψ , the QoS data are recorded in the set M = {M 1 , M 2 , ..., M k , ..., M c } of the instantaneous user-service matrices respectively collected during instants t 1 , t 2 , ..., t k , ..., t c .

A logistic function is applied on collected data to consider the data recentness impact. Indeed, the more recent the QoS values involved in the system, the more they contribute significantly to the prediction process [26] . Our decay function is defined by:

where t ∈ Ψ and γ is the decay rate. An increase of the decay rate induces a faster decrease of the decay function.

We perform a rescaling to [1, 5] in order to have a similarity to ratings that are widely used to feed recommender systems. The values obtained by the prediction process can be transformed to their original status by using the reverse scaling formula. Without loss of generality, QoS values are rescaled to the interval [1, 5] using the following function:

where Q min and Q max are respective minimal and maximal bounds of QoS values. The predicted score after the scaling step could not be the same since the scaling process is directly applied to QoS values that are thereafter embedded in the prediction formula.

In this subsection, the goal is to predict the current time slot t c . For this purpose, we are interested in the current userservice matrix M c = q s u (t c ) m×n . The matrix factorization technique is a method widely adopted in recommender systems [2] . It is based on unsupervised feature learning and consists of reducing the computational load induced by the prediction process. By applying matrix factorization, the original high-dimensional matrix M c (simply noted in M thereafter) is approximated by two low-rank matrices (see Fig.3 ).

To predict the QoS value of the current time interval, we adopt a progresive reasoning based on the following assumptions:

Assumption 1 QoS data are assumed to be static over time.

According to the matrix factorization, the QoS value q s u is approximated as follows:

where r min(m, n) is the number of latent features and r ∈ F = {1; 2; ...; f − 1; f }. The product w ur v sr approximatively assesses the interaction between a user u and a service s.

From the matrix point of view, the approximation of the user-service matrix M = q s u m×n by the matrix M is defined as follows:

where W = [w ur ] m×f and V = [v sr ] f ×n are the two lowrank matrices from factorization process. The user latent factor matrix W means the users' interest for each latent factor while V evaluates the interest aroused by each latent factor. We consider the bias b us to compensate for the QoS values variations induced by the interactions between a user u and a service s as follows [27] : where I us is the indicator function which is equal to 1 when a user u invokes a service s and 0 otherwise. By applying the bias b us , the approximation of the QoS value q s u is reformulated as follows:

From the matrix point of view, the approximative userservice matrix M is defined as follows:

The matter now is to minimize the approximation error E defined as follows: :

where is ||.|| 2 F the Frobenius norm [1] . The minimization of the approximation error of the matrix M by the matrix M is an optimization problem whose objective function is defined as follows:

where q s u = q + b u + b s + r∈F w ur .v sr .

To avoid overfitting problem, additional parameter ρ is integrated in the objective function expression as follows:

Assumption 2 QoS data are assumed to be dynamic over time.

We consider the variability of QoS values over time and the data recentness impact; the objective function is reformulated as follows: (12) where q s

The challenge now is to solve the aforementioned optimization problem. The stochastic gradient descent is a widely used as an optimization method [30] . We adopt this method to solve our optimization problem. The stochastic gradient descent is based on an iterative algorithm which updates W and V matrices following the direction of the gradient descent of the objective function. The update rules for low-rank matrices W and V are defined by (13) and (14) . 

The stochastic gradient descent process starts with the initialization of the latent features matrices W and V with random positive values. The optimal matrices W opt and V opt which minimize the objective function are obtained during the convergence of iterations carried out based on (13) and (14) . Algorithm 1 describes the latent factors unsupervised learning process.

In the next subsection, we perform the QoS Prediction for the future time slot.

AutoRegressive Integrated Moving Average (ARIMA) models are widely adopted for time series forecasting [5] . Based on the Box-Jenkins approach [3] , ARIMA models predict future values of time series by extrapolating past values. We perform the QoS prediction for the future time slot based on the ARIMA-based prediction method in [9] . The predicted value q s u (t c+1 ) of the future time interval is computed as follows:

where a and b are the orders of ARMA model; q s u (t c+1 ) is the QoS value recorded from s service invocation by u user at the future time slot t c+1 . Parameters ε(t) with t = t c−b+1 , ..., t c correspond to past and current errors independently and normally distributed following the Gaussian probability density with a mean set to 0 and variance σ 2 . Parameters φ t and θ b are respectively the autoregressive coefficient and the moving average coefficient. They correspond to parameters of Maximum Likehood Estimate (MLE), which maximize the likehood function η defined as follows:

where |Ψ | is the number of time slots in the observation window Ψ . To integrate the impact of the data recentness, we apply the decay function effect on the QoS prediction of the future time slot. For this purpose, we customize the estimate of QoS value q s u (t c+1 ) as follows:

where y(t) is the decay function.

The ARIMA based prediction is presented through a user u regarding a service s in order to deeply appreciate how the forecasting process has been performed. Thereafter, the forecasting can be extended to the whole set of users and the set of existing services. The next subsection describes the service recommendation.

The service recommendation is based on the coupling of the QoS prediction of the current time interval and the future time interval. Indeed, our method aims to guarantee the appropriateness of the services offered at the current time slot; but our approach also aims to ensure that the relevant services for the active user at the current time, that these services remain relevant in a lasting manner on the next time interval after the current time. Therefore, predicting only the current time may not be sufficient to meet the active user's expectations in a sustainable manner. To alleviate this challenge, the prediction of the future time interval aims to anticipate the future needs of the active user with the purpose of extending over time the active user's satisfaction.

The service selection lays on criteria (q c 0 , q c 0 +1 ) describing the set of recommended services S rec = {s ∈ S|q s u (t c ) ≥ q c 0 ∧ q s u (t c+1 ) ≥ q c 0 +1 }, where q c 0 and q c 0 +1 are thresholds in order to solely select high-quality services. q s u (t c ) and q s u (t c+1 ) are respectively QoS values from s service invocation by u user for the current time interval and the future time interval. By including the aforementioned steps of the prediction process, Algorithm 2 describes the service recommendation process.

The next section presents the performed experiments and results.

In this section, the performances of our method are evaluated comparatively to existing recommender systems. Experiments are performed on a real-world dataset of QoS values from web service invocations. Thereafter, TASERM's performances are studied comparatively to other recommendation methods.

The next subsection describes the experiment process. [28] have collected QoS values using WSMonitor tool to record performances during web services invocations by distributed computers from PlanetLab (https://www.planet-lab.org/). In this dataset, we use the response time and throughput performances from invocations of 4500 web services by 142 users during 64 regular time slots. Each time slot lasts 15 minutes. Thereafter, 64 instantaneous user-service matrices of dimension 142 * 4500 have been built.

The matrix from the 63rd time slot represents the current time matrix; while the matrix from the 64th time slot represents the future time matrix. The current time matrix hosts QoS values to split into test data and training data. We progressively dig the current time matrix to sketch the data sparsity effect. For this purpose, the matrix density is defined as the matrix porosity intensity; meaning the training data percentage ranging from 5% to 60% in steps of 5%. For instance, a matrix density set to 10% means that 10% of training data and 90% of test data are represented in the current time matrix. Moreover, we blend 85% of training data hosted in the current time matrix with QoS values issued from other matrices.

The next subsection presents the evaluation metrics used to study TASERM's performances.

The TASERM's performances are measured using key indicators namely the Mean Absolute Error (MAE), the Root Square Mean Error (RSME) and the Normalized Discounted Cumulative Gain (NDCG). MAE and RSME indicators evaluate the prediction accuracy. The lower values of MAE and RSME, the higher prediction accuracy. MAE and RSME values are computed as follows:

where q s u (t) is the original QoS value and q s u (t) the approximated value; N is the number of recommended services.

NDCG is widely adopted to evaluate the service ranking accuracy and is computed as follows: 

where I DCG N and DCG N respectively represent the Ideal Discounted Cumulative Gain and the Discounted Cumulative Gain of Top-N recommended services. DCG N is computed as follows:

where rel j is the QoS value related to service ranked at j position. A high NDCG N expresses a high-accurate service ranking.

The next subsection analyzes the obtained results.

For the experiments, the thresholds are set as q c 0 = 2.5, q c 0 +1 = 1.5. At the process starting, the decay rate γ is set to 0.2; the number r of factors gradually increases from 5 to 50 in steps of 5. The matrix density is set to 5% and progressively raises to 60%. In the next point, TASERM's performances are compared to those of the other recommendation methods.

Our method is compared to the following recommendation methods:

-IPCC is a recommendation method that lays on itembased CF using Pearson Correlation Coefficient (PCC) for the inter-item similarities evaluation [30] . -UPCC is a recommendation method that lays on userbased CF using PCC for the inter-user similarities evaluation. Thereafter, the prediction is performed based on the weighted average of QoS values [30] . -WSPred is a time-aware recommendation method based on a personalized QoS prediction [28] . Indeed, it can be observed that for each method, the QoS prediction is refined with the matrix density increase. However, the TASERM method significantly outperfoms others methods even in the severe data sparseness situation. Indeed, following Table 2 and Fig. 7 , MAE performance of TASERM approach shows an improvement of 37.21 % compared to IPCC, 32.61 % compared to UPCC and 30% compared to WSPred. Moreover, in Fig. 8 , RSME performance of TASERM approach displays an improvement of 35.39 % compared to IPCC, 33.40 % compared to UPCC and 30% compared to WSPred. In addition, in Fig. 9 , NDCG performance of TASERM method is 4.35% better than IPCC, 1.86% better than UPCC and 1.61% better than WSPred.

To study the impact of the number of factors r, we configure r to 5; then it gradually increases to 50 in steps of 5. We propose to study TASERM performances at matrix densities equal to 15% and 60%. Figures 10, 11 and 12 shows the impact of the number of factors. In these figures, it can be observed that TASERM's performances are maximized for r = 20. Indeed, in Figs. 10 and 11, for r = 20, MAE and RSME performance are minimal, meaning a high-accurate prediction while in Fig. 12 , NDCG trend is maximal, meaning a high-accurate service ranking. This means that for a number of factors greater than 20, the additional data constitutes noise which affects the quality of prediction.

In this paper, we proposed a data sparseness robust service recommendation approach which aims to guarantee in a sustainable way the relevance of services returned to end-users. Our method first performs the QoS prediction of the current time using a flexible factorization matrix technique applied to a set of instantaneous two-dimensional user-service matrices. Thereafter, the QoS prediction of the future time interval is performed using a time series forecasting method based on an AutoRegressive Integrated Moving Average (ARIMA) model. Based on past userside QoS observations, the QoS prediction for the current time slot indicates the service relevance in the current time while the QoS prediction for the future time slot provides information on the anticipated service relevance relatively to the active user's needs. The service recommendation is based on the pair of criteria (q c 0 , q c 0 +1 ) which aims to ensure the sustainable appropriateness of services returned to the active user. Experiments conducted on a realworld dataset show the high-performances of our method compared to competing approaches. Since the rapid growth of big data offers increased possibilities and important sources of information to integrate into the recommendation process, we plan in the future, to exploit other data sources to refine the service targeting and to get closer to end-users' expectations.

A new unbiased fir filter with improved robustness based on frobenius norm with exponential weight

Matrix factorization model in collaborative filtering algorithms: A Survey

Time series analysis: Forecasting and control

Time-aware recommender systems: A comprehensive survey and analysis of existing evaluation protocols

Timeaware cloud service recommendation using similarity-enhanced collaborative filtering and ARIMA model

Rank Correlation Methods (5th edn)

Forecasting of covid19 per regions using arima models and polynomial functions

A non negative matrix factorization for collaborative filtering recommender systems based on a Bayesian probabilistic model

Web service recommendation based on time series forecasting and collaborative filtering

Cloud service recommendation based on a correlated QoS ranking prediction

Rating-based collaborative filtering: Algorithms and evaluation

User based Collaborative Filtering using fuzzy C-means

Relational completion based nonnegative matrix factorization for predicting metabolite-disease associations

An algorithm for efficient privacy-preserving itembased collaborative filtering

Applying the learning rate adaptation to the matrix factorization based collaborative filtering

A collaborative filtering approach based on Naï Bayes classifier

Timeaware distributed service recommendation with privacypreservation

Time-Aware QoS prediction for cloud service recommendation based on matrix factorization

Timeaware music recommender systems: Modeling the evolution of implicit user preferences and user listening habits in a collaborative filtering approach

Rank Correlation Methods. By M. G. Kendall, 2nd edn

Time-Aware recommendation based on user preference driven

Personalized qos-Aware Web service recommendation and visualization

QF-RNN: QI-Matrix factorization based RNN for time-aware service recommendation

A user-Based cross domain collaborative filtering algorithm based on a linear decomposition model

Qos recommendation in cloud services

A Time-aware and data sparsity tolerant approach for web service recommendation

Matrix factorization techniques for recommender systems

WSPred: A time-aware personalized QoS prediction framework for web services

Time-aware Point-of-interest Recommendation

A survey of collaborative filtering-Based recommender systems for mobile internet applications

Qos-aware web service recommendation by collaborative filtering

Qos ranking prediction for cloud services

Functional matrix factorizations for cold-start recommendation

The authors declare that they have no conflict of interest.Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.