This is The earliest article about YouTube’s Recommendation System, originally from The YouTube Video Recommendation System. Published in 2010.

YouTube has one of the largest video recommendation systems in the world. In 2010, the whole system was relatively sketchy, but it’s worth checking out.

Goal

There are generally three purposes for users to use YouTube: 1. To watch specific videos; 2. Watch videos on specific topics. 3. Browse and find videos that interest you. The recommendation system mainly serves the third need. Therefore, the goal of the recommendation system is to recommend high-quality videos that are suitable for users’ interests, and the recommendation results should be updated with time and users’ recent behavior. Within YouTube’s recommendation app, there are some specific challenges:

  • Videos uploaded by users to YouTube generally do not have complete metadata, and many have no valuable information beyond the title.
  • Videos on YouTube tend to be short (less than 10 minutes), so user behavior is short, variable and noisy.
  • Videos on YouTube have a short life cycle.

This is the difference between the recommendation of YouTube and that of ordinary video websites, which is also the problem that many video companies will pay attention to now.

The system design

The input of recommendation system is mainly composed of content related information and user behavior related information. Among them, user behavior includes explicit and implicit feedback. The former refers to the user rating, clearly expressed like, dislike and other behaviors, the latter is browsing, watching and other behaviors. The original data also contains a lot of noise, many uncontrollable factors will affect the quality of the original data.

In order to determine which videos to recommend, the article gives the definition of relevant video: Relevant video of video V refers to the video that may be watched by users after video V. The correlation of the two videos is determined by association rule mining method. The correlation between video I and j is defined as:, includingsaidandThe number of co-occurrences,It’s a and videoandThe simplest such function is. Using this correlation coefficient can be selected with the seed videoA collection of related videos. Simply put, the video set is defined by a threshold.

Generating Recommendation Candidates

The simplest recall strategy can be obtained by combining the videos that users have watched, liked and collected as seed sets and combining their related video sets. In practice, however, these streams tend to have a narrow range, as users’ tastes tend to be consistent over time.

In order to expand diversity, YouTube chose to recall not 1-step related videos, but n-step related videos, that is, the relevant video collection obtained after n iterations of the seed video.

Ranking

After obtaining the collection of recalled videos, we need to sort these related videos. The data used for sorting mainly includes:

  • Video quality: includes views, video ratings, comments, likes and shares, etc
  • User data: The degree to which the video matches the interest of the current user
  • Diversity: There is a balance between the categories of recommended video collections to keep the results diverse

This data is eventually put together linearly to get a ranking score.

other

There are other details:

  • Recommendation reason: When recommending a video, links will be provided to explain why the video is recommended
  • The bottom layer uses BigTable and MapReduce
  • Test results using A/B test system
  • Using click through rate (CTR), long CTR (only counting clicks that led to watches of a substantial fraction of the video), session length, time until first long watch, And Recommendation Coverage (the Fraction of Logged in users with recommendations) as evaluation indexes.