This is the fourth day of my participation in Gwen Challenge

There are three main types of Redis multi-level solutions: replication, sentinel, and clustering. Today we will focus on sentinel.

What is Sentinel? What does it do?

Sentinel is a highly available solution for Redis. The Sentinel system consists of one or more sentinels to monitor the master and slave servers. When the primary server goes offline, a secondary server is automatically upgraded to the primary server.

Sentinel is essentially a special Redis server. We can start it by configuration or command.

Conf or redis-server /path/to/sentinel.conf --sentinelCopy the code

Sentinel profile

Sentinel monitor myMaster 127.0.0.1 6379 2 Sentinel Down-after-milliseconds myMaster 60000 Sentinel Fail-timeout mymaster 180000 sentinel parallel-syncs mymaster 1Copy the code

The sentinel monitor

< IP >
is the name of the primary node to be monitored (regardless of the secondary node). The number of quorum indicates how many sentinels consider the primary node to be objectively offline if they consider it subjective offline.

Internal details

What happens inside a Sentinel when it starts?

  1. Initialize the server. But because it does not use its database capabilities, it does not load RDB or AOF files.
  2. Replace the code used by the normal Redis server with the Sentinel specific code. It only allows clients to execute PING, SENTINEL, INFO, SUBSTRIBE, UNSUBSTRBE, PSUBSCRIBE, PUNSUBSCRIBE, etc. The commands like SET and EVAL are not even loaded in the command table.
  3. Example Initialize the Sentinel status.
  4. Based on the configuration file, initialize the list of primary servers monitored by Sentinel.
  5. Create a network connection to the primary server. Two asynchronous connections are created, a command connection to send and receive commands to the master server, and a subscription connection to subscribe to the “_sentinel_: Hello “channel on the master server.

Sentinel fetch message

By default:

Sentinel sends the INFO command to the primary server every 10 seconds to get the primary server information.

Sentinel sends the INFO command to the slave server every 10 seconds to get the slave server information.

Once Sentinel establishes a subscription connection with the server, it subscrires to the _sentinel_: Hello channel.

Sentinel sends a message every 2 seconds to the _sentinel_:hello channel on all monitored master and slave servers. .

Messages sent by one Sentinel are received by other sentinels and used to update their perceptions of the Sentinel that sent the message.

Checking that the Server is offline

It is divided into subjective and objective downline.

Subjective offline: Every second, Sentinel sends a PING command to all instances (primary and secondary servers, other Sentinels) with which it has created command connections to determine whether the instance is online. Returns replies other than +PONG, -loading, and -masterdown are considered subjective logoff.

Objective offline: When a Sentinel determines a server as subjective offline, it will ask other Sentinels who also monitor the server to count the number of subjective offline. When the number reaches the set value, it will be considered as objective offline.

Election Lead Sentinel

When a primary server is judged to be objectively offline, a lead Sentinel is elected to perform failover operations on the offline primary server.

Raft algorithm is adopted to implement the method of electing lead Sentinel, which is mainly summarized as first come, first served. Each Sentinel will require other sentinels to set it as a local lead, and each Sentinel has only one chance to set up a local lead. First come, first served, the highest vote gets.

reference

  1. Design and Implementation of Redis