Attention-Based Causal Representation Learning for Out-of-Distribution Recommendation

Out-of-Distribution (OOD) recommendation has emerged as a compelling field within recommender systems. Traditional recommender systems assume independent and identically distributed data between training and testing. However, real-world scenarios often deviate from this ideal, characterized by distributional shifts that render observed data incongruent with the notion of identical distribution. These discrepancies, particularly prevalent in practical applications, significantly impact the performance of recommendation algorithms, resulting in suboptimal system functionality.

Current researchs neglect the inevitable distribution shifts of users and items in practical scenarios. Addressing shifts in user features and item interactions is vital for improving recommendation systems' effectiveness. In the existing works, the Causal OOD Recommendation (COR) framework addresses out-of-distribution recommendation problem with user feature shifts from a causal perspective but primarily overlooks shifts in latent user features and intricate user preference interrelations.

To address those issues, we propose an innovative framework called Attention-Based Causal OOD Recommendation (ABCOR), which apply the attention mechanism in two distinct ways to delve into the nuances of latent user feature shifts and elucidate the complex user preference relationship network. For the shifts in potential user features, our proposed framework leverages explicitly variational attention to analyze the subtle shifts information, thereby enhancing the interaction generation process. For the complex user preference relationship network, ABCOR integrates a multi-head self-attention inference layer to elucidate and harness the complex interrelations among varied user preferences, which is conducive to learn more accurate user preference

The efficacy of our proposed approach is substantiated through rigorous validation on two public real-world datasets, wherein our models demonstrably surpass the contemporary state-of-the-art COR methodology. The ablation study findings further validate the efficacy and robustness of our proposed model excelling in accommodating shifts in user features and deciphering the dynamics of user preference relationships.

[1] SHARMA K, LEE Y C, NAMBI S, et al. A Survey of Graph Neural Networks for SocialRecommender Systems: abs/2212.04481[A/OL]. 2022. https://api.semanticscholar.org/CorpusID:254408574.
[2] GAO C, ZHENG Y, LI N, et al. A Survey of Graph Neural Networks for Recommender Systems:Challenges, Methods, and Directions[J/OL]. ACM Transactions on Recommender Systems,2021, 1: 1 - 51. https://api.semanticscholar.org/CorpusID:237940542.
[3] XIAO T, WANG S. Towards Unbiased and Robust Causal Ranking for Recommender Systems[J/OL]. Proceedings of the Fifteenth ACM International Conference on Web Search and DataMining, 2022. https://api.semanticscholar.org/CorpusID:246828659.
[4] HE X, LIAO L, ZHANG H, et al. Neural Collaborative Filtering[J/OL]. Proceedings of the 26thInternational Conference on World Wide Web, 2017. https://api.semanticscholar.org/CorpusID:13907106.
[5] SARWAR B M, KARYPIS G, KONSTAN J A, et al. Item-based collaborative filtering recom￾mendation algorithms[C/OL]//The Web Conference. 2001. https://api.semanticscholar.org/CorpusID:8047550.
[6] MUSTO C. Enhanced vector space models for content-based recommender systems[C/OL]//ACM Conference on Recommender Systems. 2010. https://api.semanticscholar.org/CorpusID:13388735.
[7] KOMPAN M, BIELIKOVÁ M. Content-Based News Recommendation[C/OL]//InternationalConference on Electronic Commerce and Web Technologies. 2010. https://api.semanticscholar.org/CorpusID:14605638.
[8] SANTANA M R O, SOARES A. Hybrid Model with Time Modeling for Sequential Recom￾mender Systems: abs/2103.06138[A/OL]. 2021. https://api.semanticscholar.org/CorpusID:232170720.
[9] BURKE R. Hybrid Recommender Systems: Survey and Experiments[J/OL]. User Modelingand User-Adapted Interaction, 2002, 12: 331-370. https://api.semanticscholar.org/CorpusID:3970.
[10] OVAISI Z, HEINECKE S, LI J, et al. RGRecSys: A Toolkit for Robustness Evaluation ofRecommender Systems[J/OL]. Proceedings of the Fifteenth ACM International Conference onWeb Search and Data Mining, 2022. https://api.semanticscholar.org/CorpusID:245877841.
[11] ARJOVSKY M, BOTTOU L, GULRAJANI I, et al. Invariant Risk Minimization:abs/1907.02893[A/OL]. 2019. https://api.semanticscholar.org/CorpusID:195820364.
[12] BLANCHARD G, DESHMUKH A A, DOGAN Ü, et al. Domain Generalization by MarginalTransfer Learning[J/OL]. Journal of machine learning research, 2017, 22: 2:1-2:55. https://api.semanticscholar.org/CorpusID:59362358.
[13] RENDLE S. Factorization Machines[J/OL]. 2010 IEEE International Conference on DataMining, 2010: 995-1000. https://api.semanticscholar.org/CorpusID:17265929.
[14] MA J, ZHOU C, CUI P, et al. Learning Disentangled Representations for Recommendation:abs/1910.14238[A/OL]. 2019. https://api.semanticscholar.org/CorpusID:202789109.
[15] WANG W, LIN X, FENG F, et al. Causal Representation Learning for Out-of-DistributionRecommendation[J/OL]. Proceedings of the ACM Web Conference 2022, 2022. https://api.semanticscholar.org/CorpusID:248367478.
[16] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-Excitation Networks[J/OL]. 2018 IEEE/CVFConference on Computer Vision and Pattern Recognition, 2017: 7132-7141. https://api.semanticscholar.org/CorpusID:140309863.
[17] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional Block Attention Module:abs/1807.06521[A/OL]. 2018. https://api.semanticscholar.org/CorpusID:49867180.
[18] WANG X, GIRSHICK R B, GUPTA A K, et al. Non-local Neural Networks[J/OL]. 2018IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017: 7794-7803. https://api.semanticscholar.org/CorpusID:4852647.
[19] FU J, LIU J, TIAN H, et al. Dual Attention Network for Scene Segmentation[J/OL]. 2019IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 3141-3149. https://api.semanticscholar.org/CorpusID:52180375.
[20] VASWANI A, SHAZEER N M, PARMAR N, et al. Attention is All you Need[C/OL]//NeuralInformation Processing Systems. 2017. https://api.semanticscholar.org/CorpusID:13756489.
[21] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of Deep Bidirectional Trans￾formers for Language Understanding[C/OL]//North American Chapter of the Association forComputational Linguistics. 2019. https://api.semanticscholar.org/CorpusID:52967399.
[22] XIAO J, YE H, HE X, et al. Attentional Factorization Machines: Learning the Weight of FeatureInteractions via Attention Networks: abs/1708.04617[A/OL]. 2017. https://api.semanticscholar.org/CorpusID:3836251.
[23] ZHOU G, SONG C N, ZHU X, et al. Deep Interest Network for Click-Through Rate Predic￾tion[J/OL]. Proceedings of the 24th ACM SIGKDD International Conference on KnowledgeDiscovery & Data Mining, 2017. https://api.semanticscholar.org/CorpusID:1637394.
[24] KANG W C, MCAULEY J. Self-Attentive Sequential Recommendation[J/OL]. 2018 IEEEInternational Conference on Data Mining, 2018: 197-206. https://api.semanticscholar.org/CorpusID:52127932.
[25] THIEBES S, LINS S, SUNYAEV A. Trustworthy artificial intelligence[J/OL]. ElectronicMarkets, 2020, 31: 447 - 464. https://api.semanticscholar.org/CorpusID:224877177.
[26] QUIONERO-CANDELA J, SUGIYAMA M, SCHWAIGHOFER A, et al. Dataset Shift inMachine Learning[C/OL]//2009. https://api.semanticscholar.org/CorpusID:61294087.
[27] SHEN Z, LIU J, HE Y, et al. Towards Out-Of-Distribution Generalization: A Survey:abs/2108.13624[A/OL]. 2021. https://api.semanticscholar.org/CorpusID:237364121.
[28] CAO L. Non-IID Recommender Systems: A Review and Framework of RecommendationParadigm Shifting: abs/2007.07217[A/OL]. 2016. https://api.semanticscholar.org/CorpusID:56098214.
[29] YANG C, WU Q, WEN Q, et al. Towards Out-of-Distribution Sequential Event Prediction: ACausal Treatment: abs/2210.13005[A/OL]. 2022. https://api.semanticscholar.org/CorpusID:253098828.
[30] HE Y, WANG Z, CUI P, et al. CausPref: Causal Preference Learning for Out-of-DistributionRecommendation[J/OL]. Proceedings of the ACM Web Conference 2022, 2022. https://api.semanticscholar.org/CorpusID:246652643.
[31] DIDELEZ V, PIGEOT I. Judea Pearl: Causality: Models, reasoning, and inference[J/OL].Politische Vierteljahresschrift, 2001, 42: 313-315. https://api.semanticscholar.org/CorpusID:141473148.
[32] NARENDRA T, SANKARAN A, VIJAYKEERTHY D, et al. Explaining Deep Learning Mod￾els using Causal Inference: abs/1811.04376[A/OL]. 2018. https://api.semanticscholar.org/CorpusID:53282816.
[33] ZHANG Y, FENG F, HE X, et al. Causal Intervention for Leveraging Popularity Bias in Recom￾mendation[J/OL]. Proceedings of the 44th International ACM SIGIR Conference on Researchand Development in Information Retrieval, 2021. https://api.semanticscholar.org/CorpusID:234482660.
[34] SCHOLKOPF B, LOCATELLO F, BAUER S, et al. Towards Causal Representation Learning:abs/2102.11107[A/OL]. 2021. https://api.semanticscholar.org/CorpusID:231986372.
[35] WANG W, FENG F, HE X, et al. Deconfounded Recommendation for Alleviating Bias Ampli￾fication[J/OL]. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery& Data Mining, 2021. https://api.semanticscholar.org/CorpusID:235166201.
[36] WANG W, FENG F, HE X, et al. Clicks can be Cheating: Counterfactual Recommendation forMitigating Clickbait Issue[J/OL]. Proceedings of the 44th International ACM SIGIR Confer￾ence on Research and Development in Information Retrieval, 2020. https://api.semanticscholar.org/CorpusID:235185402.
[37] WANG Z, ZHANG J, XU H, et al. Counterfactual Data-Augmented Sequential Recommen￾dation[J/OL]. Proceedings of the 44th International ACM SIGIR Conference on Researchand Development in Information Retrieval, 2021. https://api.semanticscholar.org/CorpusID:235792483.
[38] ZHANG S, YAO D, ZHAO Z, et al. CauseRec: Counterfactual User Sequence Synthesis forSequential Recommendation[J/OL]. Proceedings of the 44th International ACM SIGIR Con￾ference on Research and Development in Information Retrieval, 2021. https://api.semanticscholar.org/CorpusID:235792358.
[39] ZHANG Y, FENG F, HE X, et al. Causal Intervention for Leveraging Popularity Bias in Recom￾mendation[J/OL]. Proceedings of the 44th International ACM SIGIR Conference on Researchand Development in Information Retrieval, 2021. https://api.semanticscholar.org/CorpusID:234482660.
[40] ZOU H, CUI P, LI B, et al. Counterfactual Prediction for Bundle Treatment[C/OL]//NeuralInformation Processing Systems. 2020. https://api.semanticscholar.org/CorpusID:227275241.
[41] LIU D, CHENG P, ZHU H, et al. Debiased Representation Learning in Recommendation viaInformation Bottleneck[J/OL]. ACM Transactions on Recommender Systems, 2022, 1: 1 - 27.https://api.semanticscholar.org/CorpusID:253110445.
[42] XU H, XU Y, YANG Y. Causal Structure Representation Learning of Confounders in LatentSpace for Recommendation: abs/2311.03382[A/OL]. 2023. https://api.semanticscholar.org/CorpusID:265043201.
[43] WANG S, CHEN X, SHENG Q, et al. Causal Disentangled Variational Auto-Encoder forPreference Understanding in Recommendation[J/OL]. Proceedings of the 46th InternationalACM SIGIR Conference on Research and Development in Information Retrieval, 2023. https://api.semanticscholar.org/CorpusID:258180280.
[44] ZHOU C, BAI J, SONG J, et al. ATRank: An Attention-Based User Behavior Modeling Frame work for Recommendation: abs/1711.06632[A/OL]. 2017. https://api.semanticscholar.org/CorpusID:19112718.
[45] HU Q, HAN Z, LIN X, et al. Learning peer recommendation using attention-driven CNN withinteraction tripartite graph[J/OL]. Information Sciences, 2019, 479: 231-249. https://api.semanticscholar.org/CorpusID:59528813.
[46] LI J, REN P, CHEN Z, et al. Neural Attentive Session-based Recommendation[J/OL]. Pro ceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017.https://api.semanticscholar.org/CorpusID:21066930.
[47] WEN P, YUAN W, QIN Q, et al. Neural attention model for recommendation based on factor ization machines[J/OL]. Applied Intelligence, 2020, 51: 1829 - 1844. https://api.semanticscholar.org/CorpusID:225120760.
[48] KYBURG H E. Judea Pearl, Causality, Cambridge University Press (2000)[J/OL]. ArtificialIntelligence, 2005, 169: 174-179. https://api.semanticscholar.org/CorpusID:33390651.
[49] HE X, CHUA T S. Neural Factorization Machines for Sparse Predictive Analytics[J/OL]. Pro ceedings of the 40th International ACM SIGIR Conference on Research and Development inInformation Retrieval, 2017. https://api.semanticscholar.org/CorpusID:2021204.
[50] LIANG D, KRISHNAN R G, HOFFMAN M D, et al. Variational Autoencoders for Col laborative Filtering[J/OL]. Proceedings of the 2018 World Wide Web Conference, 2018.https://api.semanticscholar.org/CorpusID:3361310.