[2407.05441] Language Representations Can be What Recommenders Need: Findings and Potentials

View the PDF file for the paper entitled Language Representation can be what the recommendation needs: results and capabilities, by Leheng Sheng, Zhang, Yi Zhang, Yuxin Chen, Xiang Wang and Tat-Seng Chua

PDF HTML (experimental) view

a summary:Recent studies experimentally indicate that LMS models (LMS) codes rich global knowledge beyond the indications, and attracts great attention across various fields. However, in the field of recommendation, it remains unintentionally if LMS encodes the user preference information. Contrary to the prevailing understanding that LMS and traditional recommendations admire the surveyors of distinguished representation due to the huge gap in the goals of language modeling and behavior, this work re -examines such an understanding and explores the extraction of the recommendation space directly from the language representation space. Surprisingly, our results show that the representation of the elements, when it is written in linear LM representations, leads to a superior recommendation. This result indicates a potential symmetry between the space of the advanced language representation and the space of representation of an effective component of the recommendation, which implicitly means that cooperative signs may be implicitly coded inside LMS. Motivated by these results, we explore the possibility of designing purely COF cooperative filtering models on the basis of language representation without identity -based inclusion. To be specific, we merge several decisive ingredients to build a simple but effective model, with the addresses of the elements as inputs. Experimental results show that such a simple model can outperform the leading CF models, which shed light on the use of language representations to obtain a better recommendation. Moreover, we systematically analyze this simple model and find many of the main features of using advanced language representations: good preparation for the representation of the elements, zero recommendation capabilities, and awareness of the user’s intent. The results we have reached highlighting the relationship between language modeling and behavior modeling, which can inspire both natural language processing societies and the recommendations of the recommendation system.