On the client side, Netflix is easy to use, and the video appears as if by magic when you press the play button. Looks easy, right? But that’s not the case. Anyone familiar with cloud computing might simply assume that since Netflix uses AWS for its video service, the video will flow over the network from the stored S3 to the device the user uses to watch the video upon receiving a play request. This process makes perfect sense for a relatively small service, but Netflix doesn’t follow it exactly, and the technology involved is far more complicated and interesting than we can imagine.

Before I introduce you to these techniques, let’s take a look at some data:

  • Netflix has more than 110 million subscribers
  • Netflix operates in more than 200 countries
  • Netflix has nearly $3 billion in revenue per quarter
  • Netflix is adding more than 5 million subscribers every quarter
  • Netflix streams more than a billion hours of video every week. (For comparison, YouTube streams a billion hours in a day, compared with Facebook’s 110 million hours.)
  • Netflix streamed 250 million hours of video in a single day in 2017
  • Netflix accounts for more than 37% of peak Internet traffic in the United States
  • Netflix plans to spend $7 billion on new content in 2018

From these data, we can see that Netflix has a large number of users, who have a great demand for video playback, and every paid video service can bring revenue to Netflix. So how does Netflix keep users engaged and happy?

Netflix uses AWS and Open Connect, which work together seamlessly to provide satisfying video services for customers without time limit. Netflix can strictly control the client, back end and CDN(Content distribution network) to ensure the whole video watching experience of users from start to finish. So that users around the world can get a stable and reliable video service anytime and anywhere.

Why did Netflix choose AWS, and what are its advantages?

From building its own data center in 2007 to moving to AWS in 2008, Netflix gradually found itself focusing on strengthening its video distribution capabilities rather than building data centers. While it was a bold move to choose AWS, which was just beginning to be built, AWS just met its transformation needs. This choice is based on the following two considerations:

  • On the one hand, Netflix wants a more reliable infrastructure. It wants to remove any single point of failure in the system, and AWS happens to provide highly reliable databases, storage performance, and redundant data centers. Netflix needs cloud computing so it doesn’t have to build huge, unreliable stand-alone apps. In short, it wants to provide global services without having to build its own data center, something that the old data center couldn’t or couldn’t do.
  • The other is that Netflix no longer wants to do undifferentiated grunt work. All of this is necessary, but it is no good for the core business of providing high-quality video services. AWS does all the heavy lifting for Netflix, freeing up the company to focus on delivering core business value.

With virtually no downtime and the flexibility of the cloud, AWS, which is both reliable and cost-effective, is the perfect choice for Netflix.

What happens inside AWS before you press “Play”?

Before pressing “Play,” AWS prehandles everything else except video services, including scalable computing, scalable storage, business logic, scalable distributed databases, big data processing and analytics, recommendations, code transformation, and hundreds of other features.

  • Scalable computing and scalable storage

Scalable computing is implemented in EC2 and scalable storage is implemented in S3. Video services, from playlists to specific content, are accessed from a Netflix device (iPhone, TV, Xbox, Android phone, tablet, etc.) that accesses a computer in EC2.

  • Scalable distributed database

Netflix uses both DynamoDB and Cassandra as its distributed database. If these database names aren’t very recognizable to you, just think of them as high-quality database products.

Databases: Databases are used to store data, including configuration file information, billing information and information about movies and TV shows. Distributed: Distributed means that the database runs not just on one large computer, but on many computers. Users’ data is copied to multiple computers, so that even if something goes wrong on one or two computers, the other computers still have it, ensuring that the user’s data is safe. Scalable: Scalable means that the database can process no matter how much data the user puts into it. This is one of the main advantages of distributed databases, adding computers on demand in order to process more data.

  • Big data processing and analysis

Literally, we know that big data means having a lot of data. Netflix collects a lot of information in tracking its users’ habits, such as what they watch, where they watch it, which videos they already watch and don’t need to watch again, and how often each video is played. We call the process of unifying all data into a standard format big data processing, and reading and understanding data is called big data analysis, which is done for specific problem needs.

  • Do personalized customization for each user

How is Netflix using its data analytics capabilities to get users to watch more video? In the results screen where we retrieve videos, each video shows an image (we call it the header image). This header image may arouse the user’s interest and attention, leading the user to choose to watch the video. Needless to say, the more attractive the image at the top of a video, the more likely users are to watch it. Once you watch more videos, you’re more likely to subscribe to Netflix content.

As you can see above, it’s just Stranger Things, but Netflix has made a lot of different images for the top of the page and personalized them to different users, rather than just seeing the same image at the top. So how does it work?

Netflix randomly selects a video from a selection of images and records the number of times the video is played and the corresponding header image. In the example of Stranger Things, it’s easy to see that the picture in the middle is more attractive when the picture in the middle is viewed 1,000 times while the picture in the middle is viewed only once. Netflix would put that picture at the top of the page of Stranger Things. This is what we call data-driven selection, and Netflix is known for being data-driven.

This may seem clever, but it could be done better, just with more data, and that’s the future of learning data to solve problems. Users are different people, they may not be attracted to the same image at the top of the page, they have different tastes and preferences. Netflix understands this, which is why it’s now customizing all the images for each user, trying to pick the highlights that are most relevant to them. So how does it do that?

I want to emphasize here that Netflix not only keeps track of everything its users do on the site, it also counts. It will know the user’s favorite movies, favorite actors and so on. Let’s say Netflix recommends the movie “Good Will Hunting” to its users. The first thing Netflix does is to select a header image for its users. Its goal is to display a header image that Will interest users in this movie.

If you like comedy, Netflix will show you a Robin Williams image. If you prefer a romantic movie, Netflix will show you a picture of Matt Damon and Minnie’s driver getting ready to kiss. By showing Robin Williams, Netflix let its users know that the movie might have humorous moments, and since Netflix knows its users love comedy, this video is a good fit. The image of Matt Damon and Minnie’s driver sends a completely different message. If a user is a comedy fan and sees the image, he might skip it. This also demonstrates the importance of choosing the right image at the top of the page, as it sends a strong signal of personalization to the user (and also shows information about the movie).

Found no, Netflix’s selection of the header image will directly affect whether users watch a movie. Netflix caters to its users’ interests when it comes to selection, and it doesn’t cheat by letting users watch movies they don’t like just to get clicks. There’s no incentive. Netflix doesn’t have to pay for every episode. It tries to optimize the user experience and make them happy with what they’re watching, so it picks the best picture for the top of the page.

This is just one small example of how Netflix applies data analytics, a strategy it applies virtually everywhere.

  • recommended

How does Netflix recommend 40 or 50 videos out of hundreds of thousands? Mainly due to the use of machine learning, which is related to the big data processing and analysis we talked about earlier. Netflix uses these techniques to predict what users want to watch from its vast data set. What you actually see on screen is what Netflix has specifically chosen using machine learning.

  • Transcoding source video

Netflix needs to convert video to the format that works best for users’ devices, a process called transcoding, or encoding. Transcoding is the process of converting a video file from one format to another so that different platforms and devices can view the same video. Netflix transcodes all video in AWS, running on nearly 300,000 cpus at a time, larger than a supercomputer.

  • What does Netflix do from source media to the version you see?

Video production studios send videos to Netflix, which calls them source media and hands those new videos to the content operations team for processing. The videos are all in high definition and terabyte in size. Netflix performs a rigorous multi-step process on a video before the user sees it.

The specific steps are as follows:

Step 1: Review source Video The first thing Netflix does is spend a lot of time reviewing source video, looking for digital artifacts, color changes or missing frames that may have been caused by previous transcoding attempts or data transfer problems. Once a problem is found, the video is discarded. Step 2: Inject the media Pipeline Once the video is audited, it is injected into the media pipeline and processed in a series of steps to make the data available, like an assembly line in a factory. More than 70 processing software are used in the creation of each video. Since it is not practical to process a single multi-terabyte file directly, the first step in the media pipeline is to split the video into many smaller pieces. Step 3: Parallel code the video blocks into the pipe that is, the video blocks are processed simultaneously. That’s why Netflix is using a lot of servers in EC2. They need a lot of servers to process so many video files in parallel. Moreover, Netflix has set itself a mandatory requirement that the source media be encoded and delivered to the CDN in as little as 30 minutes. Once the blocks are encoded, Netflix revalidates them to make sure no new problems arise, then reassembles the blocks into a file and validates them again.

The immediate result is a pile of documents. In 2007, Netflix began streaming video on Microsoft Windows, and over time, Netflix began supporting more devices, including: Samsung Blu-Ray, Apple Mac, Xbox 360, LG DTV, Sony PS3, Nintendo Wii, Apple iPad, Apple iPhone, Apple TV, Android, Kindle Fire and Comcast X1. In all, Netflix supports 2,200 different devices, and with the ultimate goal of supporting every internet-connected device, it has configured a video format that works best for each device, so the coding process creates a lot of files. Netflix calls all the different formats of a video its encoding profile.

Netflix also created optimized files for different network speeds. Users will see higher-quality video on a fast network than on a slower one. There are also many files in different audio formats, and audio can be transcoded to different quality levels and different languages. There are also files for subtitles, and a video may have subtitles in many different languages. As a result, there are many versions of each video to watch, and the aforementioned Stranger Things game ended up generating 9,570 different video, audio, and text files. The choice of version to watch depends on the player’s device, network quality, Netflix plan and language.

The necessity of CDN

Once transcoding is done, how does Netflix distribute the video? Netflix has tried three different video streaming strategies: its own mini-CDNs, third-party CDNS and Open Connect. The CDN’s original purpose was to put video on servers as close to users as possible (around the world). When a user initiates a demand to watch a video, CDN finds the nearest server (with video resources) and transmits the required video source to the user’s playback device. The biggest advantages of CDN are stability and rapidity. Let’s say you’re watching a video in London, but the video stream is coming from Portland, Oregon. In this case, the video will flow through many layers of network, including unreliable underground fiber optic cables, which will directly slow down and make the transmission unreliable. It is not hard to know that the closer the video content is to the user watching it, the faster and more stable the viewing experience will be for the user.

Starting in 2011, Netflix realized that with its size, it needed a dedicated CDN solution to maximize network usage. Video distribution is its core competency and will become a huge competitive advantage. So Netflix started deploying Open Connect in 2012. For Netflix, Open Connect has many advantages, such as:

  1. It is not as expensive as third-party CDNS and can save money.
  2. Better quality, and by controlling the entire video generation path (transcoding, CDN, and the client on the device side), Netflix thinks it can provide a better video viewing experience.
  3. More scalable, Netflix’s goal is to serve customers around the world, both quickly supporting all users and providing a high-quality video viewing experience (it needs to build its own system).

Third-party CDNS must support access to any content from anywhere in the world, while Netflix does not. Netflix knows who its customers are because they have to subscribe to Netflix. Netflix also knows exactly what videos it needs to play, and when it knows it only needs to serve large video streams, Netflix can make a lot of smart optimization choices that other CDNS can’t. Netflix also knows its users’ preferences, like what videos they like to watch and when they like to watch them. So using this information, Netflix built a very efficient CDN network called Open Connect.

How does Open Connect work?

To store video, Netflix developed its own computer system. Netflix called itOCAs (Open Connect Appliances). Each OCA is a fast server, highly optimized for delivering large files, and each OCA has a number of hard drives and flash drives for storing video. OCAs entity diagram is as follows:

OCAs can be divided into several types by purpose: large OCAs can store Netflix’s entire video catalog; The slightly smaller OCAs can only store a portion of Netflix’s video catalog; Smaller OCAs are used to store videos that are actively cached during off-peak hours. OCAs are nothing special from a hardware point of view, they are all BASED on PC commodity components assembled by different vendors in specific custom cases. From a software perspective, OCAs uses the FreeBSD operating system and Nginx as Web servers, and each OCA has a Web server, while Nginx is primarily used as a video streaming server. The number of OCAs at each site depends on Netflix’s expectations of the site’s reliability, the amount of Netflix traffic (bandwidth) distributed from the site, and the percentage of traffic allowed to flow through the site.

When the user presses the play key, they see the video cached by a specific OCA (a location near the user). In order to get the best video viewing experience, Netflix even wants to cache videos in users’ homes, but that’s obviously impractical. So Netflix wanted to put mini-Netflix as close to its users as possible. How did it do it?

Netflix has a large number of video delivery points in more than 1,000 locations around the world. Netflix doesn’t run its own network, nor does it run its own data centers. Instead, Netflix puts OCAs in data centres at Internet service providers (ISPs). These OCAs are provided for free to ISPs to embed in their networks. Netflix also places OCAs at or near The Internet Switching Location (IXP). This way, Netflix can get all the benefits of someone else’s generic data center directly without having to run its own.

In addition, Netflix will proactively cache the videos that users are likely to watch every day by predicting the videos that users are likely to watch, and copy the videos that users are likely to watch into the corresponding OCAs in advance.

Summary: What happens when you press “Play”?

How exactly does Netflix offer video to its users? We made the following conclusions:

  1. The user selects the video to watch through the Netflix client on the terminal device. After pressing “Play”, the client sends a play request to the Netflix player application running in AWS (the request simply states what the user wants to play the video). It is important to mention the “play license” here, not every place every user has a play license. For some specific videos, Netflix requires an authentication to determine whether the user has “permission to play.” Creating its own content became the easiest way for Netflix to protect its users from “play licenses” around the world.
  2. Taking into account various relevant information, the playback application feeds back urls for different OCAs servers (no more than 10) of the same type that the user uses in a Web browser. Netflix uses IP addresses and information from isPs to determine which OCAs clusters are most suitable for users.
  3. The client intelligently selects which OCA to use by testing the network connection quality of each OCA, with priority given to the fastest and most reliable. Throughout the video stream processing, the client stays in a test state to detect the best way to receive content from OCA, then connects to OCA to begin transferring the video stream to the user’s device.
  4. I don’t know if you’ve noticed the difference in image quality when you’re watching a video, for example, sometimes it looks a little blurry, but after a while, it comes back to hd quality. This is because customers are adapting to the quality of the network. If the network quality degrades, the client will degrade the video quality to accommodate it. If the network quality degrades too much, the client will switch to another OCA.

Netflix: What Happens When You Press Play?

Thanks to Ding Xiaoyun for correcting this article.