Rabbit brother the following several main introduction of the content

S login cookie

■ Shopping cart cookie

■ Cache generated web pages

■ Cache database rows

■ Analyze web page visits

This article will pick up where the previous article left off with a simple use case for Redis and introduce some typical Web applications through a few examples. Although the problem shown here is much simpler than it actually is, the web application presented here can be applied directly to a real application with only minor modifications. The next main task is to serve as a practical guide to what you can do with Redis, and later articles will cover the Redis command in more detail.

At a high level, a Web application is a server or service that responds to requests sent by a Web browser over the HTTP protocol. The typical steps for a Web server to respond to a request are as follows. (1) The server parses the request sent by the client.

(2) The request is forwarded to a predefined handler.

(3) The processor may fetch data from the database.

(4) The processor render the template according to the retrieved data.

(5) The processor returns the rendered content to the client as a response to the request.

Listed in the above five steps from the Angle of the high-level shows a typical Web server works, in this case the Web request is considered to be a stateless (stess), that is, the server itself does not record any information related to the previous request, this made the server failure (fil) can easily be replaced. There are plenty of books (blog) on the various steps of the process, how to optimize the response of the rabbit elder brother the next thing to do and they are similar, the difference is that the rabbit elder brother is explained how to use a faster Redis query instead of a traditional relational database query, and how to use Redis to complete some use a relational database can’t efficient task.

The rest of the article is all about discovering and solving bugs Bunny Mall, a massive fictional online store with about 5 million different users every day who bring in 100 million hits and buy over 100, 000 items from the site. The reason why we set the data volume of Bugs Bunny Mall to be particularly large is to consider that if problems can be solved smoothly under the background of large data volume, then the problems caused by small data volume and medium data volume will not be solved. In addition, although the solutions presented in the next few articles are designed to solve the problems encountered by the large online store, all but one of the solutions can be used on a Redis server with only a few gigabytes of memory. And the goal of these solutions is to improve the performance of the system in responding to real-time requests.Copy the code

All of the solutions listed in this article (and some of their variants) have actually been used in production environments. More specifically, by transferring some of the data processing and storage tasks from traditional databases to Redis, you can speed up web page loads and reduce resource usage. The first problem we need to solve is using Redis to manage user login sessions.

Login and cookie caching

Every time we log into an Internet service, such as a bank account or email, these services use cookies to record our identity. A cookie consists of a small amount of data that a web site asks our browser to store and send back to the service each time it sends a request. There are two common ways to store login information in cookies: signed cookies and token cookies.

A signed cookie typically stores the user’s name and possibly the user ID, the last time the user was successfully logged in, and any other information the site finds useful. In addition to the user’s information, a signed cookie contains a signature that the server can use to verify that the information sent by the browser has not been changed (such as changing the login user name in the cookie to another user).

Token cookie stores a string of random bytes in the cookie as a token, and the server can find the owner of the token in the database based on the token. Over time, old tokens are replaced by new ones. The following table shows the advantages and disadvantages of signed cookies and token cookies.

Cookie type

advantages

disadvantages

Signature cookies

All the information needed to validate the cookie is stored in the cookie. Cookies can contain additional information that can be easily signed

Handling signatures correctly is difficult. It’s easy to forget to sign data, or to forget to validate the data’s signature, creating a security hole

The token cookie

Adding information is easy. Cookies are very small, so mobile terminals and slower clients can send requests more quickly

More information needs to be stored in the server. If you are using a relational database, loading and storing cookies can be expensive

Because Bunny Mall does not implement the requirement of signed cookies, we choose to use token cookies to reference the entry in the relational database table that is responsible for storing user login information. In addition to user login information, Rabbit Mall can also store users’ visit duration and the number of products viewed in the database, so that it can be analyzed in the future to learn how to better promote products to users.

In general, users often browse through several different items before deciding to purchase one or more items, and keeping track of all items viewed and when the user last visited the page often results in a large number of database writes. This user browsing data can be very useful in the long run, but the problem is that, even with optimization, most relational databases can only insert, update, or delete 200 to 2000 database rows per second on each database server. Although operations such as rush inserts, batch updates, and batch deletes can be performed at a much faster rate, high-speed batch inserts are not applicable here because the client updates only a few rows each time it visits a web page.

Because Bunny Mall’s current daily load is relatively high — about 1,200 writes per second on average, and close to 6,000 writes per second at peak times — it has to deploy 10 relational database servers to handle peak times. What we need to do is to use Redis to re-implement the login cookie function, instead of the current login cookie function implemented by the relational database.

First, we’ll use a hash to store the mapping between the login cookie token and the logged in user. To check if a user is logged in, look up the corresponding user based on the given token and return the user ID if the user is logged in. The following code shows how to check login cookies.

Inspection on the token is not difficult, because most of the complex work is done when updating the token: every time the user browse pages, program will be on the inside of the login user is stored in the hash information updates, and the user’s token and the current timestamp is added to the record recently in an ordered set of login user; If the user is viewing a product page, the program also adds the item to an ordered collection of items that the user has recently viewed, and prunes the ordered collection when the number of items logged exceeds 25. The following code shows how the program updates the token.

With the update_token () function, we can record when the user last viewed an item and which items the user recently viewed. On a server built in the last few years, the update_token () function can record at least 20,000 items per second, which is more than three times the 6,000 writes required during bugs Bunny mall’s peak. Not only that, but we can further optimize the speed of the update_Token () function by using some of the methods described below. But even this version of the update_Token () function is 10 to 100 times better than the old relational database.

Because the memory required to store session data increases over time, we need to periodically clean up old session data. To limit the amount of session data, we decided to save only the latest 10 million sessions.” Clean up the old session program consists of a cycle, the cycle of each time, will check store recent login the size of an ordered set of token, if the size of an ordered set exceeds the limit, the program will be removed from inside an ordered set of up to 100 the old token, and from the hash records user login information, Remove the information for the users for which the deleted token corresponds, and clean up the ordered collection that stores the most recent browsing history of these users. In contrast, if the number of tokens does not exceed the limit, the program sleeps for one second and then rechecks. The code below shows how to clean up the old session program.

Code listing: Clean up old sessions

Let us through calculation to find out, why this simple code can properly handle 5 million passengers a day visit, assumes that the site has 5 million users visit every day, and every user, unlike before, only need two days, the number of tokens will reach 10 million caps, and the memory space is running out of the site. Since there are 24×3600=86 400 seconds in a day, and the average website generates 500,000/86400 <58 new sessions per second, if the cleanup function runs at the rate of once per second as defined earlier in the code, it would need to clear nearly 60 tokens per second to prevent token overload. But in reality, the token cleaning function we defined can clear more than 100 tokens per second when running over the network and more than 60,000 tokens per second when running locally, which is 150-1000 times faster than the required cleanup speed, so the problem of running out of site space due to too many old tokens does not occur.

Where is the cleanup function executed? Rabbit elder brother of the article will contain some cleaning function, they may be run in a way that is daemon, may also as a regular job (cronjob) every once in a while running time, and even perform an operation at a time when running a (for example, in a acquiring a lock operation includes a cleaning operation) in general, Functions that contain while Not QUIT: code should be executed as daemons, but they can be changed to run periodically if necessary.

As you get to know More about Redis, you will gradually realize that some of the solutions shown by Bugs bunny are sometimes not the only solutions to problems. For example, in the case of the login cokie example, we could store the login user and token information directly into the string key-value pair, and then use Redis’s EXPIRE command to set the expiration time for the string and the ordered set of items that record the user’s browsing history. Redis would automatically delete them after a certain period of time. This eliminates the need to use ordered collections to keep track of the most recent tokens. However, we won’t be able to limit the number of sessions to 10 million, and we won’t be able to analyze the abandoned shopping cart after the session expires if we need to in the future.

Readers familiar with multithreaded or concurrent programming may notice that the cleanup function shown in “Code Listing: Cleaning up old Sessions” actually contains a race condition; A race condition can cause the user’s information to be mistakenly deleted if the cleanup function is deleting information about a user who is visiting the site at the same time. At present, this race condition does not significantly affect the data recorded by the application other than requiring the user to log in again, so we will leave this issue aside for the time being, and later articles will show how to prevent similar race conditions and further speed up the execution of the cleanup function.

By using Redis to record user information, we managed to reduce the number of row writes to the database by millions per day. While this is awesome, it’s just the first step in building a Web application using Redis. In the next article, Bugs Bunny will show you how to use Redis to handle another type of cookie.