Meet ARGUS: A Scalable AI Framework for Training Large Recommender Transformers to One Billion Parameters

Yandex feet Argus (Serial Modeling for Automatic Automatic User)Wide -based workforce framework for a billion recommendation systems. This penetration places Yandex between a small group of global technology leaders-along with Google, Netflix and Meta-which have succeeded in overcoming long-term technical barriers in scaling recommendation transformers.

Breaking the technical barriers in the recommendation systems

Recommendation systems have long struggled with three stubborn restrictions: short -term memory, limited expansion, and poor adaptation to user behavior. Traditional structures that cut the history of the user to a small window of modern interactions, get rid of the most famous or years of behavioral data. The result is a shallow vision of the intention that lacks long -term habits, fine shifts in taste, and seasonal courses. With the expansion of catalogs in billions of elements, these cut models are not only losing accuracy, but also suffocating the arithmetic demands for customization on a large scale. The result is familiar: meaningless recommendations, decrease in participation, and less chances to discover the coincidence.

Very few companies have succeeded in limiting the recommendation transformers beyond the experimental settings. Google, Netflix and Meta have invested extensively in this field, as it reported gains from structures such as Youtubednn, Pinnerformer and Meta. With Argus, Yandex joins a selection of companies that explain the recommendation forms of a billion teachers in live services. By modeling the entire behavioral timelines, the system reveals all of the clear and hidden connections in the user’s activity. This long horizon perspective allows argus capture developed intention and periodic patterns with much greater accuracy. For example, instead of only responding to a recent purchase, the model learns to expect seasonal behaviors – an automatic automatically for the favorite tennis brand when summer approaches – with the user demanding the repetition of the same signals year after year.

Technical innovations behind Argus

The frame offers many major developments:

• Binary target before training: Argus decomposes spontaneous learning into a shyman – Predicting the following elements and Review prediction. This mixture improves both the tradition of the historical system behavior and the model of the real user preferences.
• Development transformers symbols: Models scale from 3.2M to 1B parameters, with improvements in the performance consistent in all measures. On the scale of one billion parameters, marital accuracy increased by 2.66 %, indicating the appearance of the limitation law of the recommendation transformers.
• Extended context modeling: Argus addresses the history of users up to 8,192 long interactions in one pass, allowing customization over months of behavior instead of only the past few clicks.
• Effective polishing: The structure of two towers of the two semesters provides an unending account of the Internet from the implications and the developmentable publishing, which reduces the cost of reasoning in relation to pre -models at the level level or at the level of impression.

Spreading the real world and the gains

Argus has already been widely published on the Yandex music platform, serving millions of users. In production tests A/B, the system was achieved:

• +2.26 % increase in the total listening time (TLT)
  
• +6.37 % increased like probability

These are the largest quality improvements in the history of the platform for any model that recommends a deep learning.

Future trends

Yandex researchers are planning to expand Argus to Real -time recommendation tasksExplore An engineering feature to classify the husbandAnd adapting the frame with High -cardiac domains Like large e -commerce and video platforms. The ability to expand the range of use of the user sequence with transformer structures indicates that the recommendation systems are preparing to follow the scaling path similar to the treatment of the natural language.

conclusion

With Argus, Yandex created itself as one of the few global leaders who lead modern recommendation systems. By sharing its penetrations publicly, the company does not improve the customization only through its own services, but also accelerating the development of the entire industry recommendation.

verify Paper here. Thanks to the Yandex team to lead/ thought resources for this article.

Asif Razzaq is the CEO of Marktechpost Media Inc .. As a pioneer and vision engineer, ASIF is committed to harnessing the potential of artificial intelligence for social goodness. His last endeavor is to launch the artificial intelligence platform, Marktechpost, which highlights its in -depth coverage of machine learning and deep learning news, which is technically intact and can be easily understood by a wide audience. The platform is proud of more than 2 million monthly views, which shows its popularity among the masses.