Redis is one of the most important techniques in distributed learning. You can spend a lot of time on it

The folded wood is born at the very end; A platform of nine stories begins with soil

Redis website

High availability scheme

Part 1 – Master/slave Replication

Practical Example

  1. In /usr/local/redis/master 和 salveFolders:
mkdir -p /usr/local/redis/master/
mkdir -p /usr/local/redis/slave/
Copy the code
  1. Unzip the Redis package into the above two folders:
tar - ZXF redis - 3.2.8. Tar. Gz
cd Redis - 3.2.8
# compiler
make 
# Expect to pass all tests
make test    
Copy the code

Possible problem: GCC or TCL components are missing. Use the yum install GCC or yum install TCL command

  1. Double machine configuration

Master hindis. conf Configuration

bind 127.0.0.1 ----> bind local IP
daemonize No ----> daemonize yes(does not affect the current session, startup process hidden, daemon)
protected-mode Yes --> protected-mode no
Copy the code

Slave hinredis. Conf Configuration

bind 127.0.0.1 ----> bind local IP
daemonize No ----> daemonize yes(does not affect the current session, startup process hidden, daemon)
protected-mode Yes --> protected-mode no
port 6379 --> Port 6380
# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -
	# redis5-
slaveof The IP address of master_redis is 6379
	# redis5+
replicaof The IP address of master_redis is 6379
# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -
pidfile /var/run/redis_ 6379.pid ----> pidfile /var/run/redis_ 6380.pid
Copy the code

Linux under vim editing, you can quickly locate the field by entering/and then entering the field you want to find. Press Enter, n -> next match, n -> previous match

  1. The two-node cluster starts and connects to the client
cd / usr/local/redis/master/redis - 3.2.8 / SRC /
./redis-server  / usr/local/redis/master/redis - 3.2.8 / redis. Conf (loading configuration file)
./redis-cli -h IP -p 6379 (client connection master_redis)
cd / usr/local/redis/slave/redis - 3.2.8 / SRC /
./redis-server  / usr/local/redis/slave/redis - 3.2.8 / redis. Conf ` (loading configuration file)
./redis-cli -h IP -p 6380 '(slave_redis)
Copy the code
  1. Run the info replication command to view connection information

Host:

From under the machine:

  1. Test master/slave replication, read/write separation

Test 1: Master write slave readable

Test 2: Never write

“Basic Configuration Theory”

Redis supports primary/secondary replication, which can be enabled by executing slaveof(replicaof) or setting slaveof(replicaof) in the configuration file.

Because the persistence process of Redis cannot guarantee data loss, therefore, a single Redis cannot guarantee high availability, so we adopt the way of multi-machine and cluster to ensure its high availability.

[Pre-configuration] :

  • The main Redis configuration
    • No special configuration required
  • From Redis configuration
    • Modify the redis.conf file on the secondary server:
    # slaveof <masterip> <masterport>
# indicates that the IP address of the primary server is 192.168.10.135 and the port number is 6379
replicaof 127.0.0.1 6379
Copy the code
  • Master/slave replication
    1. Reading and writing separation: Improved overall performance and throughput (data consistency needs to be solved later)
    2. Disaster data: The host is down, and the slave machine is backed up (but the slave machine can only read but not write, and the sentry needs to be used for master/slave switchover in the later stage)

“Implementation Process Theory”

  • [Save node information]:
    • When the client sends a message to the slave serverSlaveof (replicaof) Host address (127.0.0.1) Port (6379)From the server, save the host IP address (127.0.0.1) and port (6379) toredisServer 的 masterhostAs well asmaseterport 中
    • Return THE OK from the server to the client (note: the actual replication is not performed until the OK is returned)
  • Establish socket connection:
    • Slaver establishes a socket connection with the master
    • Slaver associated file event handler
    • The processor receives RDB files (full copy) and receives write commands propagated by the Master (incremental copy)
    • After the primary server accepts the Socket connection from the secondary server, the corresponding client state is created. The slave server is the Client of the master server.
  • [Ping command]
    • Slaver sends the ping command to the Master
      1. Check the socket read/write status
      2. Check whether the Master process is normal
    • The response of the Master
      1. Send “pong”, it is normal
      2. An error is returned indicating that the Master is abnormal
      3. Timeout: indicates that the network times out
  • [Permission Authentication Configuration]:
    • If the host does not have a password (requirepass=””)
      • Do not set password on slave machine (masterauth=””)
    • If the host password (requirepass! = “”)
      • Masterauth =”Requirepass value“)
      • Or, through the machineauthCommand to send the password to the host
  • [After permission verification]:
    1. The secondary server sends the REPLCONF listening-port command to the primary serverListen on port;

    2. Synchronize data; (Redis2.8 starts with full/incremental synchronization)
    3. After the synchronization is complete, the master <–> slave enters the command propagation phase: the master server only needs to send its own write commands to the slave server, and the slave server immediately executes these write operations.

“Synchronous Data Set Theory”

  • Older versions (prior to 2.8)The synchronization function is divided into sync and command propagate.
    1. Synchronous operation:
      • By sending from the server toThe SYNCTo the master server
      • Master server generationRDB fileSend to the slave server, and send save all write commands to the slave server
      • Purge previous data from the server and perform interpretationRDB file
      • Keep data consistent (command propagation process is also required for consistency)
    2. Command propagation operation:
      • After the synchronization is complete, the primary server executes the write command. The write command is sent to the secondary server to ensure the consistency between the primary and secondary servers

    Disadvantages: There is no concept of full synchronization and incremental synchronization, the slave server will wipe all data during synchronization.

  • New version (after 2.8)You can run the PSYNC command to enable full and partial resynchronization
    • Everything else is incremental synchronization.
    • Redis full synchronization processIt can be divided into three stages:
      1. Snapshot synchronization: The Master creates a snapshot RDB and sends it to the Slave. The Slave loads and parses the snapshot. The Master also stores the new write commands generated at this stage in the buffer.
      2. Synchronous write buffer phase: The Master synchronizes the write commands stored in the buffer to the Slave.
      3. Incremental synchronization phase: The Master synchronizes write commands to the Slave.

    • The incremental synchronization
      • Incremental Redis synchronization refers to the process in which write operations performed by the Master are synchronized to the Slave when the Slave starts to work properly after initialization
      • Typically, the Master sends the same write command to the Slave for each write command that the Slave receives and executes.
  • The heartbeat detection:
    • During the command propagation phase, the slave server sends commands to the master server by default at a rate of once per second
    replconf ack <replication_offset>
# ack: Yes
# replication_offset: current replication offset from the server
Copy the code
    • The main functionAs follows:
      1. By sending the INFO replication command to the master, you can create a list of slave servers that show how many seconds have passed since the last command was sent to the master. The value of LAG jumps between 0 and 1. If the value exceeds 1, it indicates that the connection between the master and slave is faulty.
      2. Auxiliary implementation min-Slaves Redis can be configured to prevent the master server from performing write commands in unsafe situations

      Min-abolition-to-write 3 (min-replicas-to-write 3) min-abolition-max-lag 10 (min-replicas-max-lag 10) If the number of secondary servers is less than three, or the lag value of all three secondary servers is greater than or equal to 10 seconds, the primary server rejects the write command. The delay value is the lag value of the INFOreplication command.

      1. Test command lost If because of network failure, the primary server is transmitted to write command in halfway missing from the server, and then from the server to the main server sends REPLCONF ACK command, the primary server will be found from the server copy of the current offset less than their own copy offset, and then the primary server will according to submit copy of offset from the server, Find the data missing from the slave server in the replication backlog buffer and resend it to the slave server

Part 2 – Sentinel mode

“Concept”

Sentinel is a High Availability solution for Redis: A Sentinel cluster consisting of one or more Sentinel instances can monitor one or more master and slave servers. When the primary server goes offline, Sentinel can upgrade one of the secondary servers under the primary server to continue the service of the primary server, thus ensuring the high availability of Redis.

[Deployment Scheme]

“Implementing a master-slave switch instance in Sentry mode”

In part 1, we realized the master-slave replication of 6379 host and 6380 slave machine. Now, the author has added 6381 slave machine by himself.

Next, implement the sentinel cluster in the deployment scenario above:

  1. Configure master/slave sentinel.conf(the three instance sentinel configurations differ in port number, election decision threshold, and instance failure decision time)
# daemon, hidden startup, does not affect the current session
daemonize yes  
# Turn off protected mode, similar to firewall function
protected-mode no   
The default port for sentinel is 26379
port <26379 Master sentry... 26380 from Sentry 1... 26381 From Sentinel 2>
The IP address and port of the master redis monitored by the sentinel will be automatically monitored by the slave
# sentinel monitor <master-name> <ip> <redis-port> <quorum>
Quorum is a number that specifies how many sentinels consider a master to be invalid before the master is truly invalid
sentinel monitor master1 IP 6379 1
The interval between the master and the current Sentinel instance when the master is considered "failed".
# sentinel down-after-milliseconds <mastername> <millseconds>  
If there is no response or an error code within a specified period of time, then the sentinel considers the master invalid
No default 30s
sentinel down-after-milliseconds master1 3000
If no failover operation is triggered within this period after failover starts, the Current Sentinel considers the failover failure
The default time is 3 minutes
sentinel failover-timeout master1 15000
# Number of slaves that can simultaneously slaveof to the new master and SYNC when a new master is created. (The default value 1 is recommended.)
The client request will be terminated when salveof and synchronization is performed on Salve. A large value means that the sum of the time for the cluster to terminate client requests is large.
# This value is small, meaning that the "cluster" is still using old data when multiple Salves provide services to clients during a failover.
sentinel parallel-syncs master1 1
Copy the code
  1. Make sure redis is closed:
ps -ef | grep redis
Copy the code
  1. Start Redis and Sentinel in turn
. / master/redis - 6.0.8 / SRC/redis server master/redis - 6.0.8 / redis - master. Conf. / slave/redis - slave - 01 / SRC/redis server. - slave/redis-slave-02/redis-slave-6380.conf ./slave/redis-slave-02/src/redis-server Slave/redis - slave - 02 / redis - slave - 6381. Conf. / master/redis - 6.0.8 / SRC/redis - sentinel Master/redis - 6.0.8 / sentinel - master - 6379. Conf. / slave/redis - slave - 01 / SRC/redis - sentinel slave/redis-slave-01/sentinel-slave-6380.conf ./slave/redis-slave-02/src/redis-sentinel slave/redis-slave-02/sentinel-slave-6381.confCopy the code

Check thread status: (redis: one primary and two secondary, sentinel: one primary and two secondary)

After connecting to the three Sentinel clients, use info Sentinel to view the monitoring details (the monitoring details are the same as follows) :



The host is 6379 and has two slaves, monitored by three sentinels

  1. Simulated host 6379 is down

At the same time we open all sentinel.conf and find:



If 6379 is back online later, open the redis.conf file of 6379 and you will find:

“Execution Flow Theory of Sentry Mode”

Sentinel is a special Redis server that does not persist

After the Sentinel instance is started, each Sentinel will create two network connections to the primary server:

  1. Command connection: Used to send commands to the master server and receive responses
  2. Subscribe to the connection: used to subscribe to the master server- sentinel - : hellochannel

  • [Obtain master server information]

    By default, Sentinel sends the info command to the monitored primary server every 10 seconds to obtain information about the primary server and its subordinate secondary servers

    It can be used after connecting the host redis clientinfoCheck the details
  • [Obtain slave server information]

    Sentinel also establishes command connections and subscription connections to slave servers when Sentinel detects new slave servers on the primary server. Sentinel sends the info command to the slave server every 10 seconds after the command connection is established and logs the slave server’s information

  • Send messages to master and slave servers (by subscription)

    By default, Sentinel sends subscriptions to all monitored primary and secondary servers every 2s- sentinel - : helloThe channel sends messages that carry information about Sentinel itself and the master server
  • [Channel information received from master and slave servers]

    When Sentinel establishes a subscription connection with the master or slave server, Sentinel sends the following command to the server via the subscription connection: 
    The subscribe - sentinel - : helloCopy the code

    The Sentinels only create command connections with each other, but do not create subscription connections, because sentinels can sense the addition of new Sentinels by subscribes to the master or slave server. Once the new Sentinels join, the sentinels that are aware of each other communicate with each other through command connections

  • [Detecting subjective offline status]

    Sentinel sends once per second to all instances (master, slave, and other Sentinels) with which it has established command connectionsPINGAn order, if:
    1. Instance indown-after-millisecondsInvalid reply in milliseconds (except +PONG, -loading, -masterDown)
    2. Instance indown-after-millisecondsNo reply in milliseconds (timed out)

    Then Sentinel would say that instance is subjectively SDown

  • [Detecting objective offline status]

    When a Sentinel determines that a primary server is subjectively offline, Sentinel sends a query command to all other Sentinels monitoring the primary server simultaneously:
    • Host Sentinel query: 
      SENTINEL is-master-down-by-addr <ip> <port> <current_epoch> <runid>
Copy the code
    • Sentinel replies to other machines: 
      <down_state>< leader_runid >< leader_epoch >
Copy the code

    Determine if they also consider the primary server offline. If the primary server is judged to be subjective offline by all Sentinel instances that reach quorum in the Sentinel configuration, the primary server is judged to be objective offline (ODown).

  • After a primary server is deemed to be objective offline, all sentinels monitoring the primary server will select a Leader Sentinel to perform failover through raft algorithm

“Sentry Model leader Election”

  • 【Raft algorithm 】:

    Raft protocol is a protocol designed to address consistency issues in distributed systems.
    • There are three states of nodes described by the Raft protocol:Leader.Follower.Candidate.
    • Raft protocol splits time into terms, which can be thought of as “logical time”.
    • The heartbeat mechanism triggers the Leader election
    1. Leader election at initialization time
      1. After the system starts, all nodes are initialized to Follower, and term=0
      2. Node if receivedRequestVoteorAppendEntries, will maintain their Follower identity
      3. If the node is not received within a period of timeAppendEntriesIf the Leader is not found within the timeout period of the node, the Follower becomes a Candidate and starts to run for the Leader
      4. Once converted to Candidate, the node immediately starts the following things:
        • Add your own term
        • Start a new timer
        • Vote for yourself
        • Send RequestVote to all other nodes and wait for the other nodes to reply
      5. If the node receives a majority of the nodes’ consent votes before the timer expires, it becomes the Leader. Send to all other nodes at the same timeAppendEntriesAnd tell yourself that you become a Leader.

        ==> Each node can only vote one time per term, which is a first-come, first-served policy. The followers will vote for the first node that receives a RequestVote. This is the key to electing a Leader. In the same term, the node that switches to Candidate first will vote first, thus obtaining the majority of votes

    2. Leader election after the host is offline
      1. After a Sentinel determines that the master is offline objectively, the Sentinel will first check whether it has voted. If it has voted for another Sentinel, it will not become the Leader within a certain period of time
      2. If the Sentinel has not already been voted on, it becomes a Candidate
      3. Sentinel needs to do several things:
        • Update failover status to start
        • The current epoch is added by 1, which is equivalent to entering a new term (in Sentinel, the epoch is the term in Raft protocol)
        • The is-is master-down-by-addr command is sent to another node to request a vote. The command will carry its own epoch
        • Vote for yourself (leader, Leader_epoch)
      4. When other sentries receive this order, they can either approve or reject it as leader. (By judging the epoch)
      5. A Candidate keeps counting his votes until he finds that more than half of the votes approve him to be the Leader and exceeds its quorum configuration, at which point he becomes the Leader
      6. Other sentinels wait for the Leader to select the master from the slave and remove the indication of objective offline when the new master is detected to work properly
  • [Failover]:

    After Leader Sentinel is selected, the Leader Sentinel will perform a failover operation on the offline primary server in three steps:
    1. One Slave of the invalidated Master is upgraded to the new Master, and the other slaves of the invalidated Master are copied to the new Master
    2. When a client attempts to connect to an invalid Master, the cluster returns the address of the new Master to the client so that the cluster can replace the invalid Master with the existing Master
    3. After the Master and Slave servers are switched, Master’s redis.conf, Slave’s redis.conf and sentinel.conf will all have a new row of Replicaof in their redis.conf configuration files The sentinel. Conf monitoring target will be switched accordingly
  • [Primary server selection]:

    The Sentinel leader selects the new master server from the slave servers of the objectively offline master server according to the following rules
    1. Filter out nodes that are subjectively offline
    2. Select the node with the highest slave-priority. If yes, return no
    3. Select the department node with the largest replication offset, because the larger the replication offset is, the more complete the data replication will be. If yes, the data will be returned, and if no, the data will continue
    4. Select the node with the smallest run_id, because a smaller run_id means fewer restarts

Part 3 – Clustering and Partitioning

Partitioning is the distribution of data over multiple Redis instances (Redis hosts) so that each instance contains only a portion of the data

“Zoning Significance”

  1. Performance improvement:
    • The network I/O capacity and computing resources of single Redis are limited, so it is helpful to improve the overall service capacity of Redis by distributing requests to multiple machines and making full use of computing capacity and network bandwidth of multiple machines
  2. Horizontal expansion of storage capacity:
    • Although Redis service capacity can meet the application requirements, with the increase of storage data, a single machine is limited by the storage capacity of the machine itself, so the Redis service can be horizontally expanded by distributing data to multiple machines for storage

“Partition Mode”

A Partitions by area key (ID)

According to the range of ID numbers such as 1-10000, 100001-20000….. 90001-100000, each range divided into different Redis instances

  • Advantages: Simple implementation, convenient migration and expansion
  • Disadvantages: Uneven distribution of hotspot data, performance loss; Non-numeric keys such as uUID cannot be used –> {snowflake algorithm can be used instead}
B Partition using the hash algorithm

Redis instance =hash(key)%N Key: key to be partitioned, e.g. User_id N: number of Redis instances (Redis host)

  • Advantages: Supports any type of key; The hot spot distribution is uniform and the performance is good
  • Disadvantages: Complex migration, recalculation, poor scalability –> {can be solved with a consistent hash ring}

Client Partition

For a given key, the client directly selects the correct node for reading and writing. Many Redis clients implement client partitioning (JedisPool), which can also be implemented programmatically

  • [Deployment Scheme]
  • [Client selection algorithm]
    • Common hash hash(key)%N(Using hash algorithms, such as CRC32 and CRC16)

      Here’s an example: 
      user_id : u001
hash(u001) : 1844213068Redis instance =1844213068%3
// The remainder is 2, so select Redis3
Copy the code

      Advantages: Simple implementation and uniform hotspot data distribution Disadvantages: The number of nodes and keys is fixed, and the expansion requires recalculation

    • Consistency of the hash

      Normal hash modulates the number of hosts, whereas consistent hash modulates 2^32 (4 294 967 296). We think of 2^32 as a circle, like a clock, which can be understood as a circle of 60 points. Here we think of the circle as a circle of 2^32 points, as follows:



      The point directly above the circle represents 0, and the first point to the right of 0 represents 1, and so on, 2, 3, 4, 5, 6… Until 2 ^ 32-1

      So the first point to the left of 0 is 2 to the 32 minus 1. Let’s call this circle of two to the thirty-second pointsHash ring

      Suppose we have three cache servers, server A, server B, and server C. Then, in the production environment, these three servers must have their own IP addresses. We use their RESPECTIVE IP addresses for hash calculation, and use the hash result to modulo 2^32.Hash (server IP address) % 2^32

      The result of the above formula must be an integer between 0 and 2^32-1. We use this integer to represent server A, server B, and server C. Since this integer must be between 0 and 2^32-1, there must be A point on the hash ring in the figure above that corresponds to this integer. That is, server A, server B, and service C can be mapped to the ring:



      Assuming that we need to cache data using Redis, we have the following formula:Hash (key) % 2^32



      Due to theThe cached object has a fixed hash value with the server, so, under the condition of invariable in the server, the data will be cached to fixed on the server, so the next time you want to access the data, as long as using the same algorithm to calculate again, which can calculate this data is cached on the server, find the corresponding data can be directly to the corresponding server

      Hash ring migration:

      We ideally map the three servers evenly to the hash ring. So the range of data is 2^32/N. But that is often not the case. A server may have a large number of data and a small number of data, resulting in uneven server performance. This phenomenon is called hash ring offset:



      Theoretically, we can reduce the offset by adding servers, but the cost is too high, so we adopt virtual nodes to solve the problem:

      Disadvantages:

      1. High complexity: Clients have to deal with routing, high availability, failover, etc. Using partitions, data processing can become complicated, having to deal with multiple Redis databases and AOF files, and not having to persist your data between multiple instances and hosts
      2. Hard to expand: If a node is added or deleted, the key cannot be matched in Redis. Therefore, the node must be re-calculated and all or part of the data must be manually migrated

Proxy Partition

A proxy or proxy cluster is introduced on the client and server. The client sends commands to the proxy, and the proxy routes the commands to the corresponding server according to the algorithm. Common proxies are Codis (Pea pod) and TwemProxy (Twitter)

Codis was opened source by Wandoujia in November 2014 and developed based on Go and C. It is one of the outstanding open source software developed by Chinese recently.

Codis feature blog writing at……

Official Cluster partition ☆

After Redis3.0, Redis officially provides a complete clustering solution. The solution uses decentralized methods, including Sharding, replication, and failover. Called RedisCluster

Before Redis5.0, use Redis-trib to create and manage clusters. After Ruby supports Redis5.0, you can directly use redis-CLI to create and manage clusters

  • [Deployment Architecture]

  • [Decentralization] RedisCluster consists of multiple Redis node groups. It is a P2P cluster architecture with no central node
  • 【Gossip Protocol 】

    The Gossip protocol is a communication protocol, a way of spreading messages (originating from the spread of viruses). The basic idea is as follows:
    • A node selects randomly selected nodes periodically (per second) and passes information to them. The nodes that receive the information then do the same thing, passing it on to some other randomly selected node; Information is periodically transmitted toNTarget nodes. thisNReferred to asfanout(Fan out).
    • Gossip protocolstype:
      1. meet: Sender sends a request to the Receiver to join the sender’s cluster
      2. ping: Node checks whether other nodes are online
      3. pong: receiver Receives the reply message after the meet or ping. Pong is also broadcast by the new Master after failover
      4. fail: After node A determines that node B is offline, node A broadcasts the FAIL message of node B. Other nodes that receive the fail message mark node B as offline
      5. publish: Node A receives A publish command. Node A executes the command and broadcasts the publish command to the cluster. All nodes that receive the publish command execute the same publish command

      Through the Gossip protocol, a cluster can provide important cluster functions such as synchronizing status updates between clusters and election self-service failover

  • 【 solt 】

    A redis-cluster maps all physical nodes to slots from 0 to 16383 in an average or continuous allocation mode

    As shown in the deployment architecture above, there are five primary nodes, so that when a RedisCluster is created, slots can be allocated as follows:
    1. Redis1: 0-3270
    2. Redis2: 3271-6542
    3. Redis3: 6543-9814
    4. Redis4: 9815-13087
    5. Redis5: 13088-16383

    When a key-value needs to be placed in the Redis cluster, The Redis first uses crC16 algorithm to calculate a result for the key, and then takes the remainder of the result for 16384, so that each key will be corresponding to a hash slot with the number between 0 and 16383. Redis maps hash slots to different nodes roughly equally based on the number of nodes

  • [Cluster Building Example]:

    A RedisCluster requires at least three primary servers and three secondary servers

    1. 6* Cluster-enable yes in redis. Conf
    2. Write batch filesstart.shStart 6 Instances of Redis at once: 
      echo "=== "7001 start ==="
cdredis-7001/src/ ./redis-server .. /redis.confecho "=== 7001 Start end ==="
echo "7002 start start ==="
cd. /.. /cdredis-7002/src/ ./redis-server .. /redis.confecho "=== 7002 start end ==="
echo "7003 start start ==="
cd. /.. /cdredis-7003/src/ ./redis-server .. /redis.confecho "=== 7003 Start end ==="
echo "=== "7004 start ==="
cd. /.. /cdredis-7004/src/ ./redis-server .. /redis.confecho "=== 7004 Start end ==="
echo "=== "7005 start ==="
cd. /.. /cdredis-7005/src/ ./redis-server .. /redis.confecho "=== 7005 Start end ==="
echo "7006 start start ==="
cd. /.. /cdredis-7006/src/ ./redis-server .. /redis.confecho "=== 7006 Start end ==="
Copy the code

      And usechmod u+x start.shAssign write and execute permissions

    3. Start and check the thread:

    1. Create a Redis cluster

      Enter redis on any port (7001 for example) and run the following command: 
      /redis-7001/ SRC /redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7003 127.0.0.1:7005 127.0.0.1:7002 127.0.0.1:7004 127.0.0.1:7006-1 - cluster - replicas#The semantics of this command are as follows: To create a cluster, press'1'There are six host addresses in total. 1 to 1 indicates that the first three host addresses are primary and the last three host addresses are secondary
Copy the code



      Now let’s look at what we created beforenodes.conf

    2. The client connects to the cluster 
      #-c indicates the connection in cluster mode/redis-cli -h 127.0.0.1 -p 7001 -cCopy the code

      You can usecluster nodesCommand to instantly view the matching relationship between nodes and slots

  • “Shard”:
    • Client routing:

      Compared with stand-alone Redis, the client of Redis Cluster needs to be able to identify route semantics and have certain route caching capabilities
    • Version redirection:
      1. Each node communicates by sharing the relationship between slots in the Redis Cluster and corresponding nodes in the Cluster
      2. The client sends a command to any node in the Redis Cluster. The node that receives the command hashes to 16384 based on CRC16 rules to calculate its slot and corresponding node
      3. If the slot for storing data is allocated to the current node, the command is removed from the slot and the command execution result is returned to the client
      4. If the slot where the data is stored is not managed by the current node, the moved redirection exception is returned to the client
      5. The client receives the result returned by the node. In the case of the Moved exception, the client obtains the information about the destination node from the Moved exception
      6. The client sends a command to the target node to obtain the command execution result

      In the last example we had another 7001 set that was redirected to 7005 which is the ‘Moved’ redirect

    • Ask a redirect:

      To expand or reduce the capacity of a cluster, migrate slots and data in slots

      When the client sends a command to a node, the node returns moved to the client

      If the cluster is expanding or empting, when the client sends a command to the correct node, the slot and the data in the slot have been migrated to another node, it will return Ask. This is the Ask redirection mechanism

      1. The client sends a command to the target node, and the slot in the target node has been migrated to another node. At this time, the target node returns ask to the client
      2. The client sends the Asking command to the new node, and then again to the new node
      3. The new node executes the command and returns the command execution result to the client
    • ‘Moved’ is different from ‘ask.:
      • Moved: The slot has been transferred
      • Ask: The tank is still in the process of transfer
    • Smart Smart client —JedisCluster:

      JedisCluster is an intelligent cluster client provided by Jedis according to the features of RedisCluster
      • JedisCluster creates a connection pool for each node and a Cluster slots mapping cache for each node.
      • JedisCluster establishes a mapping cache for each slot of the master node and the connection pool of the master node
      • When JedisCluster starts, the relationship between keys, slots, and nodes is known, and the destination node can be found
      • JedisCluster sends commands to the destination node, and the destination node responds directly to JedisCluster
      • If JedisCluster fails to connect to the destination node, JedisCluster knows that the connected node is the wrong one, and the node returns a moved exception to JedisCluster
      • JedisCluster reinitializes the cache relationship between slot and node, and then sends commands to the new target node. The target command executes the command and responds to JedisCluster. If the command is sent more than five times, Throws the exception “Too many cluster redirection!”
      • JedisPoolConfig config = new JedisPoolConfig();
Set<HostAndPort> jedisClusterNode = new HashSet<HostAndPort>();
jedisClusterNode.add(new HostAndPort("XXX.XXX.XXX.XXX".7001));
jedisClusterNode.add(new HostAndPort("XXX.XXX.XXX.XXX".7002));
jedisClusterNode.add(new HostAndPort("XXX.XXX.XXX.XXX".7003));
jedisClusterNode.add(new HostAndPort("XXX.XXX.XXX.XXX".7004));
jedisClusterNode.add(new HostAndPort("XXX.XXX.XXX.XXX".7005));
jedisClusterNode.add(new HostAndPort("XXX.XXX.XXX.XXX".7006));
JedisCluster jcd = new JedisCluster(jedisClusterNode, config);
jcd.set("user:1:name"."Archie");
String value = jcd.get("user:1:name");
Copy the code
  • “Migration”:

    The node corresponding to each slot in a RedisCluster is determined after initialization.

    In some cases, nodes and shards need to change:
    • New nodes are added as master
    • A node group needs to go offline
    • Slot distribution needs to be adjusted for load imbalance

    At this point, the fragmentation needs to be migrated, and the trigger and process control of migration are completed by the external system. Contains the following two types:

    • Node migration status setting: Mark source/destination nodes before migration
    • Key migration atomization command: specific steps of migration

    1. Send the status change command to node B, and set the slot state of node B to Importing
    2. When migrating, the system sends A status change command to node A to change the status of slot A to MIGRATING
    3. Send the migrate command to A to tell A that the key corresponding to the slot to be migrated will be migrated to B
    4. After all keys are migrated, cluster setSlot resets the slots
  • [expansion]:
    • Create two new Redis instances with no data – 7007&7008
    • Modify the base configuration as before 7001
    • Start the service for 7007./redis-7007/src/redis-server /redis-7007/redis.conf
    • Add 7007 node as new node and start 
      /redis-cli --cluster add-node 127.0.0.1:7007 127.0.0.1:7001Copy the code

    • After logging in to any client, you can view the node allocation information of the current clustercluster nodes

    • Hash Slot redistribution (data migration)
      1. Connect to the Cluster
      The. / SRC/redis - cli - cluster reshard 127.0.0.1:7007Copy the code
      1. Enter the number of slots to be allocated
      How many slots do you want to move (from 1 to 16384)? Input "3000"Copy the code
      1. Enter the node ID of the receiving slot
      What is the receiving node ID? ==> "Enter the ID of 7007"Copy the code
      1. Enter the source node ID (enter all)
      Please enter all the source node IDs. 
	Type 'all' to use all the nodes as source nodes for the hash slots. 
	Type 'done' once you entered all the source nodes IDs.
Copy the code
      1. Type yes to start moving the slot to the target node ID
      Do you want to poceed with the proposed reshard plan (yes/no)? Input "yes"Copy the code
      1. View the results
      cluster nodes
Copy the code

    • Add 7008 to secondary node 7007:
      1. Start the 7008 service
      2. Add slave node to 7008 using redis-cli: 
        / SRC /redis-cli --cluster add-node 127.0.0.1:7008 127.0.0.1:7007 --cluster-slave --cluster-master-id a1e298448ae98d96e98dc59103c7d68a3930788bCopy the code

      3. Log in to any REIDS client to view the nodes in the cluster

  • 【 shrinkage capacity 】:
    • Command:. / redis - cli - cluster del -node 192.168.127.128:7008 48 d7a4697f89065c1b70297f530d3ba3be27a733

    [ERR] Node 192.168.127.128:7008 is not empty! Reshard data away and try again. You need to allocate the hash slot occupied by this node

  • [Disaster Recovery failover]:
    • Fault detection:

      Each node in the cluster sends PING messages to other nodes in the cluster periodically (every second). If node A does not receive A pong response from node B within A certain period of time (cluster-node-timeout), node A identifies node B as PFAIL.

      If the number of PFail marks of B is more than half (N/2 + 1) of the primary nodes in the cluster, B is marked as Fail. A reports to the whole cluster that the node is offline.

      Other nodes receive broadcasts and mark B as FAIL.
      • [Slave node election] : Each slave node sets an election time based on its offset of replicated data to the master. The slave node with a larger offset (more replicated data) has a higher election time and is preferred for election. The slave sends a FAILVOER_AUTH_REQUEST message to other masters. The master sends a FAILOVER_AUTH_ACK message to tell the slave whether to accept the request.

        Before sending the FAILOVER_AUTH_REQUEST message, the slave increates the currentEpoch and adds the latest Epoch to the FAILOVER_AUTH_REQUEST message. If the slave has not voted for the currentEpoch, the slave responds with the currentEpoch. Otherwise, the slave replies with the currentEpoch

        All the masters start voting for the slave election, voting for the slave to vote, if most of the mastersThe node (N / 2 + 1)If they all vote for a slave node, then the election passes, and that slave node can be switched to master

        Determination of RedisCluster failure:

        1. More than half of the primary nodes in the cluster are down (unable to vote)
        2. The secondary node of the primary node is down (slot allocation is not continuous)
    • Change notification:

      When a slave receives approval from more than half of the masters, it becomes the new master. At this time, it will broadcast itself as master through PONG message with the latest Epoch, so that other nodes in the Cluster can update topology (node.conf) as soon as possible.
    • Master-slave switch:
      1. Automatic switchover: the above mentioned secondary node election
      2. Active failover: Manual failover is an expected operation rather than an actual failure. The purpose is to swap roles between the current master node and one of the slave nodes (the nodes that perform cluster-failover) in a safe manner (without data loss)
        1. Sending a cluster failover command to a slave node (slaveof no one)
        2. The slave node tells its master node to make a manual switch (CLUSTERMSG_TYPE_MFSTART)
        3. The master node blocks all client commands (10s)
        4. The slave node obtains the replication offset of the master node from the ping packet of the master node
        5. Copy from node to offset, initiate an election, count votes, win an election, upgrade to master node, and update configuration
        6. After the switch is complete, the original master sends the Moved instruction to all clients to redirect to the new master

        If the primary node is offline, cluster Failover Force or Cluster Failover Takeover is used to forcibly switch over the primary node

    • A copy of the drift:

      We know that in the case of one master and one slave, if both master and slave fail, the whole cluster fails. In order to avoid this situation, we can do more than one, but then the cost increases.
      • Redis provides a method called replica drift, which can improve cluster reliability without adding too many slave machines

      Master1Downtime, thenSlaver11Promoted to new Master1; The cluster detected a new oneMaster1Is a single point, so the cluster starts from the node group with the most slaves (Master3), select the slave with the least alphabetical node name (Slaver31) drifts to a single master-slave node group (Master1)

      1. Delete Slaver31 slave records from Master3
      2. Change the host of Slaver31 to Master1
      3. Add Slaver31 as the slave node in Master1
      4. Change the copy source of Slaver31 to Master1
      5. Ping packets are used to synchronize information to other nodes in the cluster