Redis has been widely used in various Internet architecture scenarios, and its excellent performance, good operability and a large number of scenarios have attracted much attention. In this paper, the author introduces a disaster recovery solution of Redis in the non-large cluster distributed application scenario. Take a lookCopy the code

A brief introduction to Redis

Redis is a high performance key-value non-relational database. Because of its high performance characteristics, it supports high availability, persistence, a variety of data structures, clustering, etc., making it stand out and become a commonly used non-relational database. In addition, Redis has many usage scenarios.

  1. Session Cache Redis Cache sessions have a very good advantage because Redis provides persistence, which can provide a good support for long sessions in application scenarios that require long sessions, such as shopping cart scenarios, to provide users with a good shopping experience.

  2. Full page caching exists in WordPress, and Pantheon has a great plugin called WP-redis that loads pages you’ve ever seen as quickly as possible.

  3. Queue Reids provide list and set operations, which makes Redis a good platform for message queuing.

    We often do purchase limits through the queuing functionality of Reids. For example, during holidays or promotion period, some activities should be carried out to limit users’ purchase behavior, limiting them to purchase several times today or one time in a period of time. Also more suitable for application.

  4. Ranking Redis does a very good job of incrementing or decrementing numbers in memory. Therefore, we will apply Redis in many ranking scenarios. For example, the novel website will rank novels and recommend the top novels to users according to the ranking.

  5. Publish/subscribe Redis provides publish and subscribe functions, publish and subscribe scenarios are many, for example, we can based on publish and subscribe script triggers, with Redis publish and subscribe functions built up chat system.

There are plenty of other scenarios that Redis does well.

Second, single point of failure in the use of Redis

Just because Redis has a variety of excellent and special features, and the application scenarios are very rich, so that Redis has its presence in various companies. Then comes the problem and the risk. Although Redis has rich application scenarios, some companies still use relatively conservative single-node deployment in the practice of Redis, which brings security risks for future maintenance.

In 2015, I dealt with a business interruption caused by a single point of failure. At that time, Redis did not adopt distributed deployment, adopted single-instance deployment, and did not consider the problem of disaster recovery.

At that time, we controlled users’ purchasing behavior of preferential products through Redis server. But later, due to unknown reasons, the server of Redis node went down, so we could not control users’ purchasing behavior, resulting in users’ purchasing behavior of preferential products for several times in a period of time.

This kind of downtime has caused irreparable losses to the company, and the security risk is very serious. As the operator of the system at that time, it is necessary for me to repair the problem and improve the architecture. So I began to learn about Redis single point of failure in non-distributed applications.

3. Backup and disaster recovery of Redis applications in non-distributed scenarios

Redis master-slave replication should be common now. There are two common master-slave replication architectures.

Redis is commonly used for master/slave replication

  • Plan aThis is the most common architecture, with one Master node and two Slave nodes. The client writes data to the Master node and reads data to two slaves. In this way, the read expansion is realized and the read load of the Master node is reduced.

  • Scheme 2This architecture is also one Master and two slaves. The difference is that Master and Slave1 use keepalived for VIP transfers. The Client connects to the Master through the VIP. Scenario 1 IP changes are avoided.

Advantages and disadvantages of Redis master-slave replication

  • advantages

  1. If the master fails, the slave node can be promoted to a new master and continue to provide services for the old master

  2. Implement read extensions. The master-slave replication architecture is generally used for read extensions. The Master writes data while the Slave reads data

  • insufficientArchitecture Option 1When the Master is faulty, the Client is disconnected from the Master and cannot write data. In addition, the Slave cannot copy data from the Master.

In this case, perform the following operations (assuming Slave1 is promoted to Master):

1) Run the slaveof no one command on Slave1 to upgrade Slave1 to the new Master node. 2) Make slave writable on Slave1 because in most cases slave is read-only. 3) Tell the Client (the program that connects to Redis) the connection address of the new Master node. 4) Configure Slave2 to copy data from the new Master.

Framework Option 2When the master fails, the Client can connect to Slave1 for data manipulation, but Slave1 becomes a single point of failure, which is often avoided.You need to perform the following operations:

1) Run the slaveof no one command on Slave1 to upgrade Slave1 to the new Master node. 2) Configure Slave1 to be writable because in most cases the Slave configuration is read-only. 3) Configure Slave2 to copy data from the new Master

It can be found that both architectural solutions require human intervention to perform failover. The need for manual intervention increases the workload of operation and maintenance, and has a huge impact on the business. This is where Redis’s highly available solution, Sentinel, can be used

Four, Redis Sentinel introduction

Redis Sentinel offers a high availability solution for Redis. As a practical matter, using Redis Sentinel allows you to create a Redis environment that can prevent certain failures without human intervention. Redis Sentinel is designed as a distributed architecture that runs multiple Sentinel processes to work together. When multiple Sentinels can no longer serve a given master, fault detection is performed, which reduces the possibility of false positives.

5. Redis Sentinel function

The main functions of Redis Sentinel in Redis high availability scheme are as follows:

  • Sentinel constantly checks to see if the master and slave are working as expected

  • Notification Through the API, Sentinel is able to notify system administrators that Redis instances monitored by the program have failed

  • Automatic failover If the master does not function as expected, Sentinel can initiate a failover process in which one slave is promoted to master and the other slaves are reconfigured to use the new master. Applications using the Redis service, when connected, You will also be notified to use the new address.

  • The configuration provider Sentinel can serve as the authentication source for client service discovery: the client connects to Sentinel to get the Redis Master address currently responsible for a given service. If a failover occurs, Sentinel reports the new address.

Six, Redis Sentinel architecture

Seven, the realization principle of Redis Sentinel

The Sentinel cluster monitors its own and Redis master/slave replication. If the Master node is faulty, perform the following steps:

  • 1) Elections are conducted between sentinels, and a leader is elected to perform failover

  • 2) Sentinel leader selects one of the slave nodes as the new Master node. To elect a slave, perform the following operations:

    a) Disconnect time with masterIf the disconnect time from the master exceeds down-after-milliseconds(sentinel) * 10 seconds plus the time it took to failover from the time sentinel determined that the master was not available, The slave is considered unfit to be promoted to master. b)Slave priorityEach slave has a priority, which is stored in the redis.conf configuration file. If the priorities are the same, proceed.C) Replication offset positionThe replication offset records where the data is copied from the master. The larger the replication offset is, the more data is received from the master. If the replication offset is the same, proceed with the election.Run IDThe process for electing the Slave with the minimum Run ID as the new Master is as follows:

  • 3) The Sentinel leader will perform slaveof no one operation on the new master elected in the previous step to promote it to the master node

  • 4) The Sentinel leader sends commands to other slaves to make the remaining slaves become the slaves of the new master node

  • 5) The Sentinel leader will demote the original master to slave. When the master works normally, the Sentinel leader will send commands to make it copy from the new master. All the above failover operations are completed by Sentinel itself without manual intervention.

conclusion

High availability of Redis is achieved with Sentinel, and failover can be achieved without human intervention when master fails. This avoids impact on services and improves O&M efficiency. When deploying Sentinel, it is recommended to use an odd number of Sentinel nodes, with a minimum of three sentinel nodes.

Write in the last

Because sentinel knowledge is more, here is just to introduce you, let you have an understanding.

, end,

— If it helps, share it with your friends —

Long press concern public number