key: cord-024430-r0gbw5j6
authors: Wang, Hao; Shen, Huawei; Cheng, Xueqi
title: Modeling Users’ Multifaceted Interest Correlation for Social Recommendation
date: 2020-04-17
journal: Advances in Knowledge Discovery and Data Mining
DOI: 10.1007/978-3-030-47426-3_10
sha: 
doc_id: 24430
cord_uid: r0gbw5j6

Recommender systems suggest to users the items that are potentially of their interests, by mining users’ feedback data on items. Social relations provide an independent source of information about users and can be exploited for improving recommendation performance. Most of existing recommendation methods exploit social influence by refining social relations into a scalar indicator to either directly recommend friends’ visited items to users or constrain that friends’ embeddings are similar. However, a scalar indicator cannot express the multifaceted interest correlations between users, since each user’s interest is distributed across multiple dimensions. To address this issue, we propose a new embedding-based framework, which exploits users’ multifaceted interest correlation for social recommendation. We design a dimension-wise attention mechanism to learn a correlation vector to characterize the interest correlation between a pair of friends, capturing the high variation of users’ interest correlation on multiple dimensions. Moreover, we use friends’ embeddings to smooth a user’s own embedding with the correlation vector as weights, building the elaborate unstructured social influence between users. Experimental results on two real-world datasets demonstrate that modeling users’ multifaceted interest correlations can significantly improve recommendation performance.

Recommender systems suggest to users the items that are potentially of their interests [22] by mining users' feedback data on items [23] . Real-world recommender systems often allow users to build social relations [26] , and such social relations provide an independent source of information about users beyond the feedback information [13] . Social correlation theories [18] , such as homophily and social influence, indicate that there are correlations between two socially connected users [2] , which can potentially be used to exploit social relations for improving recommendation accuracy [25] .

Many methods have been proposed for social recommendation in recent years, and these methods can be mainly grouped into two categories: (1) memory-based methods [1, 12, 14] use social relation as an indicator that filters relevant users and directly recommend friends' visited items to a user; (2) model-based methods [4, 5, 9, 10, 22, 27, 29, 31] integrate social relation into factorization methods to constrain that friends share similar interest embeddings. Moreover, feedbackbased similarities are utilized to weigh friends' interest relevance in memorybased methods [3] or embedding coherence in model-based methods [11, 17] . In sum, existing methods refine two users' social relation into a scalar indicator to build their interest correlation. However, each user's interest is differently distributed across multiple dimensions, and the consistency in one dimension does not mean consistency in other dimensions. As Fig. 1 shows, a user's interest similarities with his friends vary greatly in different categories of items. When the user needs suggestions on one category, he may refer to friends with strong correlations on that category, and suggestions of friends with strong correlations on other categories are not useful. Therefore, a global scalar indicator used in existing methods cannot express the multifaceted interest correlations between friends. Unfortunately, there exists no explicit evidence to refine social networks into the elaborate correlation, and simply distinguishing items' categories would make the problem of data sparsity even more serious, which is not conducive to the learning of model parameters.

In this paper, we propose a new embedding-based social recommendation method (Fig. 2) . We propose to use a correlation vector, instead of a scalar value, to characterize the interest correlation between each pair of friends, and design a dimension-wise attention mechanism with the social network as input to learn it. The correlation vector has the same dimension with user's embedding, thus can sufficiently capture the high variation in users' interest correlations on each fine-grained dimension. Moreover, we smooth a user's embedding by his friends' embeddings, with the correlation vector into consideration. The combination of the dimension-wise attention mechanism and the smoothing operation can impose strong and delicate unstructured correlations on users' embeddings while making interactions between users and items. Such an end-to-end framework allow the proposed method to learn the unstructured correlations in a fully data-driven manner.

We evaluate the proposed method by extensive experiments on two realworld datasets collected from Gowalla and Epinions respectively. Experimental results show that the proposed method outperforms the state-of-the-art social recommendation methods.

Recommender systems normally utilize the user-item rating information for recommendation. Collaborative filtering [7] has become one of the most popular technologies, which achieves good recommendation results.

Social Recommender Systems leverage the social network information to enhance traditional recommendation methods [6, 21, 30, 32] . According to the nature of the existing social recommendation techniques, we classify them into two main categories: memory-based methods [1, 12, 14] which normally directly or indirectly recommend users items that their friends like, and model-based methods [4, 5, [9] [10] [11] 22, 27, 29, 31] use users' social relations to constrain that friends share similar embeddings.

In sum, existing methods use a scalar value to build friends' interest correlation, which cannot sufficiently express their multifaceted interest correlations. Although Yang et al. [28] integrate items' category information to train a matrix factorization model for each category of items, they make data too sparse to learn parameters, and they cannot utilize correlations among different categories.

Attention Mechanism has recently been used in recommendation tasks [15, 16, 24] . For example, Sun et al. [16] use attention to model the dynamic social influence for recommendation. However, they still express users' interest correlation by a scalar value, which cannot sufficiently capture the high variation of users' interest correlation.

For ease of description, we first formalize variables used and the problem dealt with in this paper. We denote with U and I the set of users and the set of items respectively. For a user u and an item i, we denote with r ui u's feedback to item i. We use S to represent the social network over users in U . S u represents user u's friend set and S uv = 1(0) indicates whether there exists social relation between user u and user v. In our model, we learn a preference vector t u for each user u and a preference vector z i for each item I.

Item Recommendation: Given a set of users U with social relations S, a set of items I and U 's feedback over items I, item recommendation recommends for each target user u ∈ U a list of items {i|i ∈ I} consiting of items that the target user is potentially interested in and has not interacted with them up to the recommendation.

Next, we present the proposed Dimension-wise Attention model for Social Recommendation, i.e., DASR in Fig. 2 .

Users' interest is often differently distributed across multiple dimensions. To accurately capture friends' influence on user's preference, one needs to model the multi-dimensional interest correlation between users. Fortunately, attention mechanism seems to provide a feasible solution, since it can automatically models and selects pertinent piece of information with the attentive weights from a set of inputs, where higher (lower) weights indicate the corresponding inputs more informative to generate the outputs. To accommodate our problem, we further design a dimension-wise attention mechanism and use it to learn a correlation vector for each pair of friends, building their multi-dimensional interest correlation for social recommendation. Figure 2 shows the architecture of DASR, which includes an attention layer, a smoothing layer and a recommendation layer. We learn a preference vector for each user and item, namely, t u and z i , and use DASR to infer a target user u's preference to a candidate item i with social influence of u's friends into consideration. Instead of directly performing inner product between t u and z i in the recommendation layer, we first use embeddings of user u's friends to smooth user u's own embedding in the smoothing layer, and the smoothing weights are the correlation vector learned in the attention layer. With these designs, we can learn strong and delicate unstructured correlations of users' embeddings in a fully data-driven manner and provide better item recommendation.

We input the embeddings of the target user u and users in his friend set S u to the attention layer, and compute the interest correlation vector between user u and each friend v.

We first use a weight matrix W a to perform self-attention on user u and friend v as follows:

where t u and t v are embeddings of user u and friend v. T represents transposition and || denotes the concatenation operation. LeakyReLU (x) = max(0, x) + βmin(0, x) acts as non-linear activation function with β as the negative slope. e uv is the attention coefficient that indicates the importance of friend v's features to user u. To ensure that e uv can express the correlation of each dimension in user's embeddings, we design W a as a weight matrix with dimension 2d * d.

For user u, we get an interest correlation matrix C u , and each column of C u represents the correlation vector between user u and one of his friends. To make coefficients easily comparable across different friends, we normalize C u 's each row across all choices of v. We denote with α uv the normalized interest correlation vector between user u and friend v:

For each friend v in user u's friend set S u , we obtain a normalized correlation vector α uv to represent dimension-wise interest correlation between user u and user v. We then smooth user u's embedding by adding each friend v's embedding with the correlation vector α uv serving as smoothing weight.

where h u is user u's smoothing embedding. is the element-wise Hadamard product and σ(z) = 1 1+e −z offers nonlinearity. h u consists of both user u's and his friends' embeddings, allowing the smoothing embedding not only to retain user u's own unique interest, but also to integrate his friends' interest. In this way, we can learn different patterns of each user's interest correlation, e.g., some users barely refer to their friends, while some users often refer to a few friends' suggestions, etc.

In the recommendation layer, we use user's smoothing embedding, i.e., h u , to make recommendation.

Denote p ui as user u's preference to item i, and we compute p ui as follows:

where z i denotes item i's embedding. We define two types of objective functions according to the feedback type, including implicit feedback, e.g., users' check-in counts at POIs, and explicit feedback, e.g., users' rating scores to items. First, we define the objective function in a ranking manner. For each positive feedback (u, i), we randomly select c negative samples from item set I with item i excluded and denote the set of negative samples as NEG(i). The objective function is defined as follows:

The ranking-based objective function can be applied to both explicit feedback and implicit feedback. Second, we define a square error-based objective function for explicit feedback only, in order to predict a user's rating score to an item:

where y ui is the user u's true rating score to item i. Finally, for user u, we compute his preference to each item in I according to Eq. 4, and take top n items as the recommendation list.

We use two real-world datasets collected from Gowalla [20] and Epinions [12] respectively for evaluation.

Gowalla is a Location-Based Social Network (LBSN), and we utilize users' check-in at Point-of-Interests (POIs) and the social network to make POI recommendation [19] . There are 1,196,248 check-ins generated by 18,737 users over 32,510 POIs in the Gowalla dataset. The total number of users' friendship records is 86,985. Epinions is a general consumer review site where users can review items. Different from Gowalla with a two-way connetions between users, users' social relationship in Epinions is their Web of Trust, which is a one-way connection, like Twitter followings. We adapt our model to this different structure, and utilize users' rating histories and trust network to make item recommendation [12] . The Epinions dataset consists of 49, 290 users who rated a total of 139, 738 different items at least once. The total number of reviews is 664, 824. The total number of issued trust statements is 487, 181.

For each user u, we partition his feedback set into three parts, i.e., 70% as training data, 15% as validation data, and 15% as testing data.

To evaluate the recommendation performance, we use two widely-used metrics on both datasets, namely, precision@n and recall@n, where n is the number of items in the recommendation list. They are computed as follows:

where P n u is the set of top n items in user u's recommendation list, and T u is user u's ground truth set of items. |x| denotes the cardinality of set x. For each metric, we consider 4 values (i.e., 1, 5, 10, 20) of n in our experiments.

For Epinions, we also evaluate the prediction on users' explicit rating scores with MAE, and it is computed as follows:

where y ui is the true ratings given by user u for item i, and abs(·) is the absolute value function.

Many existing methods are available for POI recommendation, and it is impossible to list all of them as baselines. Here, we select the baselines which serve as representative works of memory-based and model-based social recommendation methods. The baselines include:

-SoCF [3] : SoCF is a social-based collaborative filtering method, which recommend friends' visited items to users. -SoReg [11] : SoReg defines individual-based regularization with Pearson Correlation Coefficient (PCC) in traditional matrix factorization model. The PCC-version regularization achieves the best performance, compared with other variants, as reported in [11] . -LOCABAL [17] : LOCABAL takes advantage of both local friends and users with high reputations for social recommendation. -PTPMF [22] : PTPMF is a probabilistic matrix factorization model that incorporates the distinction of strong and weak ties. -ASMF/ARMF [8] : ASMF and ARMF argument user-item matrix using friends' visited items as potential items. ASMF optimizes a square-loss based matrix factorization model with potential items being assigned a score lower than a user's own visited items. ARMF optimizes a ranking-based matrix factorization model which assumes that users' preference to different items are: visited items > friends' visited (potential) items > unvisited items.

In the experiments, we add a L 2 regularization term to the users' and items' embeddings when performing optimization, and the regularization coefficient is set as 0.001. We set the negative slope β of the LeakyReLU function as 0.2. For all latent vectors, we set their dimension as n = 128. We set the negative count c as 10. The learning rate decreases from an initial value of 1.0 with the increase of iterations, and the decay factor is set as 0.5. Fig. 4 present the precision@n and recall@n of all methods in comparison on the Gowalla dataset and the Epinions dataset respectively. It can be observed that the proposed DASR method achieves the best performance under different settings of n on both datasets and both metrics, which demonstrates the superiority of our method to these state-of-the-art methods. We take Fig. 3 as an example to make a detailed discussion. Specifically, among these methods, SoCF is the only memory-based method, which directly recommend friends' items to users. Performance of SoCF is worse than other model-based methods, which learn users' and items' embeddings. SoReg integrates social relations as regularization term in matrix factorization model with feedback-based similarities as regularization coefficients, leading to that friends share similar embeddings. It achieves a good result. Besides social relations as local context, LOCABAL exploits extra social influence, i.e., users with high reputation as global social context. This makes LOCABAL outperform SoReg. PTPMF splits social relations as strong ties and weak ties, and distinguish the different influence of the two types of social ties. We can observe that the differentiation in PTPMF model benefit the recommendation performance. Since ARMF's performance is better than ASMF, we present ARMF only for comparison. It is observed ARMF is the best baseline method. This may profit from that ARMF introduce friends' visited items as potential items and it optimizes users' preference to items in a ranking manner. The proposed method, i.e., DASR, learns rich correlation patterns between users' interest by a correlation vector and finally beats these baselines. Different from Fig. 3 , we can observe that LOCABAL is better than PTPMF on the Epinions dataset in Fig. 4 . This indicates that weak ties in Epinions dataset do not provide valuable suggestions for users.

Feedback in the Epinions dataset is users' rating scores for items. We also present rating prediction results of different methods in comparison, as shown in Fig. 5 .

Note that, we use ASMF, rather than ARMF, since ASMF is a square loss-based method and ARMF focuses on item ranking.

It is observed that the proposed DASR achieves the best MAE metric on the Epinions dataset. By comparing all methods, we can find that the results of rating prediction is similar to those of recommendation results. The difference lies in that ARMF occupies the second best position in the comparison of recommendation results, while it is slightly better than SoCF only in the comparison of rating prediction. The reason may be: the value assigned to potential items cannot accurately express users' true preference and factorization on these potential values makes parameter learning deviate from a better direction. 

We learn an interest correlation vector for each pair of users with social relations. Each dimension in the correlation vector represents the interest correlation in the same dimension of users' embeddings. In what follows, we study the interest correlation patterns between different pair of friends. For ease of exhibition, we set the dimension of users' embeddings as 32 and train a new DASR model to get the attention weights between each user and his friends. We select two users from our Gowalla dataset, and both them have 11 friends. Denote the two users as u 1 and u 2 respectively, we draw the heat map of the weights of attention vectors with their friends. Figure 6 shows u 1 's (left) and u 2 's (right) attention weights. For each user, we compute the norms of his correlation vectors with friends, and rearrange friend id in the descending order of the norm.

We have the following observations: (1) Most dimensions of the attention vector between u 1 and friend 0 are large values, which indicate that they are very similar and we can recommend friend 0's visited items to u 1 . (2) Each dimension of the attention vector between u 1 and friend 11 is a small value, which indicates they have no similar interest and recommending friend 11's visited items to u 1 cannot achieve a good performance. (3) By comparing u 1 and u 2 , we can find that each user's interest correlations with his friends have a specific patterns. u 1 may mainly refer to suggestions of several friends with very strong interest correlation, while u 2 may refer to suggestions of each friend dispersedly.

These observations demonstrate that the proposed method outperforms baselines, and modeling users' multi-dimensional interest correlation can significantly improve recommendation performance.

In this paper, we propose a new embedding-based social recommendation method. We use a correlation vector to characterize the high variation of users' interest correlations on all dimensions, and design a dimension-wise attention mechanism to learn the correlation vector. Moreover, we use a user's friends' embeddings to smooth the user's embedding with the correlation vector as weights, and build strong and delicate unstructured social influence. Experimental results on two realworld datasets collected from Gowalla and Epinions respectively demonstrate the superiority of our method to state-of-the-art methods.

Trust based recommender system for the semantic web

Modeling users' exposure with social knowledge influence and consumption influence for recommendation

Personalized recommendation of social software items based on social relations

Social recommendation with interpersonal influence

A matrix factorization technique with trust propagation for recommendation in social networks

Social recommendation with cross-domain transferable knowledge

An automatic weighting scheme for collaborative filtering

Point-of-interest recommendations: learning potential check-ins from friends

Learning to recommend with social trust ensemble

SoRec: social recommendation using probabilistic matrix factorization

Recommender systems with social regularization

Trust-aware recommender systems

PsRec: social recommendation with pseudo ratings

Trust in recommender systems

Session-based social recommendation via dynamic graph attention networks

Attentive recurrent social recommendation

Exploiting local and global social context for recommendation

Social recommendation: a review

ULE: learning user and location embeddings for POI recommendation

Exploiting POI-specific geographical influence for point-of-interest recommendation

Joint topic-semantic-aware social recommendation for online voting

Learning personalized preference of strong and weak ties for social recommendation

Social recommendation with strong and weak ties

Social recommendation with optimal limited attention

Network embedding based recommendation method in social networks

NPA: neural news recommendation with personalized attention

Social collaborative filtering by trust

Circle-based recommendation in online social networks

Factorization vs. regularization: fusing heterogeneous social relationships in top-n recommendation

Dual influence embedded social recommendation

Leveraging social connections to improve personalized ranking for collaborative filtering

User preference learning for online social recommendation