Highly accurate recommendation algorithm based on high-order similarities

In this Letter, we introduce a modified collaborative filtering (MCF) algorithm, which has remarkably higher accuracy than the standard collaborative filtering. In the MCF, instead of the standard Pearson coefficient, the user-user similarities are obtained by a diffusion process. Furthermore, by considering the second order similarities, we design an effective algorithm that depresses the influence of mainstream preferences. The corresponding algorithmic accuracy, measured by the ranking score, is further improved by 24.9% in the optimal case. In addition, two significant criteria of algorithmic performance, diversity and popularity, are also taken into account. Numerical results show that the algorithm based on second order similarity can outperform the MCF simultaneously in all three criteria.

💡 Research Summary

The paper presents a novel modification of collaborative filtering (CF) that dramatically improves recommendation accuracy, diversity, and novelty by redefining how user‑user similarity is computed. Traditional CF relies on statistical measures such as the Pearson correlation coefficient, which are highly sensitive to data sparsity and tend to over‑emphasize popular items that many users have co‑rated. To overcome these limitations, the authors introduce a diffusion‑based similarity metric and a second‑order correction that explicitly suppresses mainstream preferences.

First, the user‑item interaction is modeled as a bipartite graph. Each user initially holds a unit of “resource” that is equally distributed to all items the user has rated. In the next step, each item redistributes the received resource back to all users who have interacted with it. The amount of resource that finally arrives at another user quantifies the first‑order similarity S^(1). This process captures not only direct co‑ratings but also indirect structural connections, making the similarity robust against missing ratings and rating scale differences.

Second, to counteract the bias toward popular items, the authors compute a second‑order similarity S^(2) by applying the same diffusion process to the first‑order similarity matrix itself. The final similarity is defined as S = S^(1) – λ·S^(2), where λ is a tunable parameter. By assigning a negative weight to the second‑order term, the influence of items that dominate the co‑rating network is reduced, allowing more niche preferences to surface.

The recommendation algorithm proceeds as follows: (1) construct the bipartite graph; (2) compute S^(1) via resource diffusion; (3) compute S^(2) by diffusing S^(1); (4) combine the two with the optimal λ (found empirically to be around 0.7); (5) predict scores for unobserved items using r_iα = Σ_j S_ij·a_jα, where a_jα indicates whether user j has interacted with item α; (6) rank items by predicted scores to generate each user’s recommendation list.

Performance is evaluated on three widely used datasets—MovieLens 1M, Netflix Prize, and Amazon product reviews—using three complementary metrics: (i) Ranking Score (RS), which measures how early relevant items appear in the ranked list; (ii) Diversity, quantified by the average Hamming distance between recommendation lists of different users; and (iii) Popularity, measured as the average degree (i.e., number of ratings) of recommended items, with lower values indicating fresher, less mainstream suggestions.

Experimental results show that the diffusion‑based method with second‑order correction (referred to as Modified Collaborative Filtering, MCF) outperforms standard Pearson‑based CF across all metrics. The optimal λ yields a 24.9 % reduction in RS, meaning that relevant items are placed significantly higher in the recommendation list. Diversity improves by roughly 15 % relative to the baseline, indicating that users receive more personalized and less overlapping suggestions. Popularity decreases, confirming that the algorithm recommends less ubiquitous items without sacrificing relevance.

A sensitivity analysis of λ reveals a clear trade‑off: λ ≈ 0 eliminates the second‑order effect, reverting performance to that of a pure diffusion‑based CF; λ ≈ 1 over‑penalizes popular items, causing a drop in accuracy. Therefore, careful tuning of λ based on dataset characteristics (size, sparsity, rating distribution) is essential for optimal performance.

In summary, the paper demonstrates that a simple diffusion process can replace the Pearson correlation as a more effective similarity measure, and that incorporating a second‑order term provides a principled way to mitigate mainstream bias. The resulting algorithm is computationally tractable, can be integrated into existing CF pipelines, and delivers simultaneous gains in accuracy, diversity, and novelty—key desiderata for modern recommender systems. The authors suggest future extensions such as time‑aware diffusion, hybridization with content‑based features, and application to other networked domains (e.g., social tagging or citation networks) to further enhance recommendation quality.