Workflow of recommendation system

In the modern society with the rapid development of the Internet, people are bombarded with hundreds of pieces of information every day, such as APP push, news hotspot, information flow advertisement… An effective “information filter” has become a critical requirement in People’s Daily life, and it is also a vital skill for information suppliers to stand out in the fierce market environment.

Recommendation systems are acting as sieves to help us find what we want most. However, the high technical threshold and research and development cost of recommendation system block many enterprises in the door. The intelligent recommendation products launched by the Fourth Paradigm based on machine learning technology focus on personalized recommendation of the content industry and effectively solve this problem with its own technical advantages. It has served 36 krypton, petal, fruit shell and other well-known media and has been widely praised in the industry.

In the following article, the recommendation system will systematically explain the relevant knowledge of the recommendation system, and I hope you can have more and more understanding of the recommendation system. First, we’ll start with the workflow of the recommendation system.

1. Information collection

This phase collects information about the user, such as user attributes, user behavior, or resources accessed by the user, to generate user profiles for the prediction task. The recommender system will not run until the user profile is fully established. Recommendation systems need to know as much as possible about the user so that they can provide the user with reasonable recommendations from the beginning.

Recommendation systems rely on different types of input, such as the most direct explicit feedback, in which users directly enter content of interest, or implicit feedback, in which preferences are indirectly inferred by observing user behavior, and mixed feedback can be obtained through a combination of explicit and implicit feedback.

Taking online learning platform as an example, user portrait is a collection of personal information associated with a specific user. This information includes the user’s cognitive skills, intelligence level, learning style, interests and interactions. User portraits are usually used for information retrieval during user model construction. In other words, the user portrait is a rough reflection of the user model. To make a successful recommendation system, it largely depends on its ability to represent users’ interests. An accurate user model is essential for accurate recommendation results.

1.1 Explicit feedback

Web sites typically prompt users to comment on content in the user interface in order to build and improve the user model for that user. The accuracy of recommendation results depends on the number of ratings provided by users. The more user ratings, the more accurate the recommendation results. The only downside to explicit feedback is that it relies heavily on the motivation of users to rate, and users are not always willing to rate. However, in contrast, display feedback does not involve the step of obtaining user preferences from user behaviors, so it provides more reliable data and the whole recommendation process is more transparent, which can better perceive the quality of the recommendation system and improve user satisfaction.

1.2 Implicit feedback

The background of the website automatically predicts the user’s interest preference by monitoring the different behaviors of the user, such as purchase history, navigation history, the time spent on certain web pages, the links, buttons and email content that the user clicks. Implicit feedback deduces user preferences from user behavior, reducing the rating burden on users. Implicit feedback is less aggressive and less accurate in user rating.

There are also some people who believe that the data provided by users’ implicit feedback is actually more objective. In the case of implicit feedback, users do not need to react in the way expected by the public, nor do they have any need to maintain their self-image, so the data provided is more authentic.

1.3 Mixed feedback

The advantages of implicit and explicit feedback can be combined in a hybrid system to minimize the shortcomings of both and achieve the best performance of the recommender system. Specifically, use implicit feedback data to validate explicit feedback data, or allow explicit feedback only when users express explicit interest.

2. Algorithm learning stage

In this stage, the system will filter user feedback obtained in the previous stage through learning algorithm and extract user characteristics. The details of this section will be covered in a future article.

3. Prediction/recommendation phase

At this stage, the system predicts the type of content the user is likely to like. This step can be done directly based on the data set collected during the information collection phase (memory – or model-based) or through user behavior monitored in the background.

Workflow of recommendation system

Stay tuned for more details on the recommended filtering techniques in the next article.