Introduction to the Redis cluster
Redis Cluster is a distributed storage solution officially released by Redis 3.0. Fully decentralized, consisting of multiple nodes, all connected to each other. The Redis client can connect to any node directly to obtain a key pair in the cluster without the need for an intermediary agent. If the node does not have a key specified by the user, the client is automatically redirected to the node where the key is located.
A Redis cluster is a network structure where each node is connected to each other via TCP connections. In a cluster of N nodes, each node has n-1 outgoing TCP connections, and n-1 incoming connections, and these TCP connections remain permanent.
Like other distributed storage systems, Redis Cluster provides the following functions:
Data partition
Redis cluster will disperse user data to each node, exceeding the maximum storage capacity of single Redis memory. The cluster introduces the concept of hash slots. After the establishment of hash slots, 16,384 hash slots will be generated, and 16,384 hash slots will be mapped to different nodes roughly equally based on the number of nodes. When a user stores a key-value, the cluster performs CRC16 verification on the key and then takes the module of 16384 to determine which slot to place the key-value in. In this way, data is automatically divided to different nodes.
Data redundancy
Redis clusters support master/slave replication and failover. The cluster uses a master-slave replication model. Each master node must have at least one slave node. If a primary node fails, all its children broadcast a packet to the other primary nodes in the cluster to request votes. Once a secondary node receives a majority of responses from the primary nodes, it wins the election and is elected as the primary node to handle the hash slot that the old primary node used to handle.
For detailed introduction and implementation principle of Redis Cluster, please refer to Redis Cluster tutorial and Redis Cluster specification. https://redis.io/topics/cluster-tutorialhttps://redis.io/topics/cluster-spec
Download & install Redis
Experimental environment information
CentOS Linux release 7.4.1708Redis version 5.0.3
Install a stand-alone Redis on the server or VM. If the Redis has been installed, you can skip this section.
Access the Redis installation directory.
cd /usr/localCopy the codeDownload and unzip the Redis source code package.
Wget http://download.redis.io/releases/redis-5.0.3.tar.gz tar - ZXVF redis - 5.0.3. Tar. GzCopy the codeThen go to the decompressed directory and use the make command to compile and install Redis.
cdRedis-5.0.3 make && make installCopy the codeDon’t be happy, because it is highly likely that you will run into compile failures because the GCC compiler is not installed. Take your time and execute the following commands in sequence.
yum -y install gcc
make distclean
make && make installCopy the codeRedis is developed based on C language, so compiling source code requires GCC (a compiler under Linux, which is used to compile. C files) support. If no, run the yum -y install GCC command to install the GCC compiler, make distclean to clear the generated files, and make && make install to recompile and install the GCC compiler.
If output similar to the following is displayed, Redis is successfully installed.
. Hint: It's a good idea to run 'make test';) INSTALL install INSTALL install INSTALL install INSTALL install INSTALL install make[1]: Leaving the /usr/local/redis-5.0.3/src directoryCopy the codeIf the source code is correctly compiled and executed, the make install command will install the program to the specified directory, usually /usr/local/bin. You can verify the installation using the following command output: Of course, you can also use the make install PREFIX=<path> command to install to the specified path.
[root@localhost bin]# cd /usr/local/bin
[root@localhost bin]# ls -l32672-rwxr-xr-x. 1 root root 4367328 3月 6 06:11 redis-benchmark -rwxr-xr-x. 1 root root 8092024 3月 6 06:11 1 root root 8092024 3月 6 06:11 redis-check-rdb-rwxr-xr-x. 1 root root 4802696 3月 6 06:11 1 root root 12 3月 6 06:11 redis-sentinel -> redis-server-rwxr-xr-x. 1 root root 8092024 3月 6 06:11 redis-serverCopy the codeAt this point, the standalone Redis installation is complete. Interview questions about Redis: Common Redis interview questions
Set up the Redis cluster
Let’s get down to business.
According to the internal failover implementation principle of Redis Cluster, a Redis Cluster needs at least three master nodes, and each master node has at least one slave node. Therefore, a Cluster should contain at least six nodes, three master nodes and three slave nodes, and be deployed on different machines.
The conditions are limited. In the test environment, we can only create a pseudo cluster on one machine and start multiple Instances of Redis through different TCP ports to form a cluster.
At present, there are two ways to build Redis Cluster:
- Manually set up a cluster by running the cluster command.
- In automatic mode, the cluster management tool is used to quickly set up the cluster.
The principle of the two methods is the same. The automatic construction method only encapsulates the Redis command to be executed in the manual construction method into an executable program. In the production environment, you are advised to use the second method, which is simple and easy to error. However, both methods will be mentioned in this practical demonstration.
Manual setup
Start node
The first step in building a cluster is to start up each node that participates in the cluster.
Since we are simulating multiple nodes on one machine, we can plan the attributes of each node in advance:
Based on the previous plan, you can run the following command to create a directory for storing the startup configuration file of each node.
mkdir /usr/local/redis-cluster
cd redis-cluster
mkdir -p 7001 7002 7003 8001 8002 8003Copy the codeRun the following commands in sequence to go to the Redis source package directory and copy the default configuration file Redis
cd /usr/localCp/redis - 5.0.3 redis. Conf/usr /local/redis-cluster/7001
cp redis.conf /usr/local/redis-cluster/7002
cp redis.conf /usr/local/redis-cluster/7003
cp redis.conf /usr/local/redis-cluster/8001
cp redis.conf /usr/local/redis-cluster/8002
cp redis.conf /usr/local/redis-cluster/8003Copy the codeNext, you need to modify the configuration files for each node individually. The following are required parameters that are enabled or modified in the /usr/local/redis-cluster/7001/redis.conf configuration file of node A. For other nodes B, C, D, E, and F, refer to the modification, and modify the ports as planned for each node.
bind 192.168.83.128 Set the host address of the current node
port 7001 Set the client connection listening port
pidfile /var/run/redis_7001.pid Set the Redis instance PID file
daemonize yes Run the Redis instance as a daemon
cluster-enabled yes Enable cluster mode
cluster-node-timeout 15000 Set the number of milliseconds for the current node connection timeout
cluster-config-file nodes-7001.conf Set the path of the current node cluster configuration fileCopy the codeYou can start the six nodes in the cluster using the following set of commands.
/usr/local/bin/redis-server /usr/local/redis-cluster/7001/redis.conf
/usr/local/bin/redis-server /usr/local/redis-cluster/7002/redis.conf
/usr/local/bin/redis-server /usr/local/redis-cluster/7003/redis.conf
/usr/local/bin/redis-server /usr/local/redis-cluster/8001/redis.conf
/usr/local/bin/redis-server /usr/local/redis-cluster/8002/redis.conf
/usr/local/bin/redis-server /usr/local/redis-cluster/8003/redis.confCopy the codeFinally through the ps – ef | grep redis command confirmed that each node service is already running.
[root@localhost bin]# ps -ef|grep redis
root 5613 1 0 04:25 ? 00:00:00 /usr/local/bin/redis-server 127.0.0.1:7001 [cluster] root 5650 1 0 04:26? 00:00:00 /usr/local/bin/redis-server 127.0.0.1:7002 [cluster] root 5661 1 0 04:26? 00:00:00 /usr/local/bin/redis-server 127.0.0.1:7003 [cluster] root 5672 1 0 04:27? 00:00:00 /usr/local/bin/redis-server 127.0.0.1:8001 [cluster] root 5681 1 0 04:27? 00:00:00 /usr/local/bin/redis-server 127.0.0.1:8002 [cluster] root 5690 1 0 04:27? 00:00:00 /usr/local/bin/redis-server 127.0.0.1:8003 [cluster] root 5731 1311 0 04:28 PTS /0 00:00:00 grep --color=auto redisCopy the codeThe above output shows that the six nodes planned above are started successfully.
Node to shake hands
Although cluster support is enabled for the above six nodes, they are not trusted or connected by default. Node handshake is the creation of links between nodes (each node is connected to other nodes) to form a complete grid, or cluster.
The command for node handshake is as follows:
cluster meet ip portCopy the codeBut to create a cluster, you don’t need to send all the cluster meet commands needed to form a full grid. As long as enough cluster meet messages can be sent so that each node can reach each other through a series of known nodes, the missing links will be created automatically.
For example, if we connect node A to node B and B to C via cluster meet, A and C will find the handshake and create the link themselves.
The six nodes we created can be connected to node A through redis- CLI to execute the following five commands to complete the handshake. In the production environment, IP 127.0.0.1 needs to be replaced with an external IP address.
Cluster meet 127.0.0.1 7002 cluster meet 127.0.0.1 7003 cluster meet 127.0.0.1 8001 cluster meet 127.0.0.1 8002 cluster Meet 127.0.0.1 8003Copy the codeIf the preceding command is executed normally, the following output is displayed:
[root@localhost bin]# /usr/local/bin/redis-cli -p 7001
127.0.0.1:7001> cluster meet 127.0.0.1 7002
OK
127.0.0.1:7001> cluster meet 127.0.0.1 7003
OK
127.0.0.1:7001> cluster meet 127.0.0.1 8001
OK
127.0.0.1:7001> cluster meet 127.0.0.1 8002
OK
127.0.0.1:7001> cluster meet 127.0.0.1 8003
OKCopy the codeYou can run the Cluster Nodes command to check the link status between nodes. I randomly found two nodes B and F to test, and the output results are as follows.
[root@localhost /]# /usr/local/bin/redis-cli -p 7002 cluster nodes61 e8c4ed8d1ff2a765a4dd2c3d300d8121d26e12 127.0.0.1:7001 @ 17001 master - 0 1552220691885 4 connected a8a41694f22977fda78863bdfb3fc03 dd1fab1bd 127.0.0.1:8002 @ 18002 master - 0 1552220691000 5 51987 c4b5530c81f2845bb9d521daf6d3dce3659 connected 127.0.0.1: master 8001 @ 18001-0 1552220690878 3 connected 1 b4b3741945d7fed472a1324aaaa6acaa1843ccb 127.0.0.1:7002 @ 17002 Myself, master 1552220690000-0 1 connected 19147 f56e679767bcebb8653262ff7f56ca072a8 127.0.0.1: master 7003 @ 17003-0 1552220691000 2 connected ed6fd72e61b747af3705b210c7164bc68739303e 127.0.0.1:8003 @ 18003 master - 0, 1552220690000 connected [root@localhost /]# /usr/local/bin/redis-cli -p 8002 cluster nodes1 b4b3741945d7fed472a1324aaaa6acaa1843ccb 127.0.0.1:7002 @ 17002 master 1552220700255-0 1 connected Ed6fd72e61b747af3705b210c7164bc68739303e 127.0.0.1:8003 @ 18003 master - 0, 1552220703281 connected 19147 f56e679767bcebb8653262ff7f56ca072a8 127.0.0.1:7003 @ 17003 master - 0 1552220700000 2 connected a8a41694f22977fda78863bdfb3fc03 dd1fab1bd 127.0.0.1:8002 @ 18002 myself, master - 0 1552220701000 5 61 e8c4ed8d1ff2a765a4dd2c3d300d8121d26e12 connected 127.0.0.1:7001 @ 17001 master - 0 1552220702275 4 connected 51987 c4b5530c81f2845bb9d521daf6d3dce3659 127.0.0.1:8001 @ 18001 master - 0 1552220701265 3 connectedCopy the codeIt can be seen that nodes B and F have been linked to the other five nodes respectively.
At this point, the node handshake is complete.
Distribution of slot
At this time, the Redis cluster is not online. You can run the cluster info command on any node to check the running status of the cluster.
[root@localhost ~]# /usr/local/bin/redis-cli -p 7001 cluster info
cluster_state:fail
......Copy the codeCluster_state :fail indicates that the cluster is offline. Because a cluster cannot go live until slots (hash slots, as mentioned in the first section of this article) are assigned to all the primary nodes in the cluster.
The commands for assigning slots are as follows:
cluster addslots slot [slot ...]Copy the codeBased on the pre-planning, this step requires using the Cluster addslots command to manually allocate 16,384 hash slots roughly equally to primary nodes A, B, and C.
/usr/local/bin/redis-cli -p 7001 cluster addslots {0.. 5461} /usr/local/bin/redis-cli -p 7002 cluster addslots {5462.. 10922} /usr/local/bin/redis-cli -p 7003 cluster addslots {10923.. 16383}Copy the codeAfter the preceding three commands are executed, you can view some running parameters of the current cluster again.
[root@localhost ~]# /usr/local/bin/redis-cli -p 7001 cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:5
cluster_my_epoch:4
cluster_stats_messages_ping_sent:11413
cluster_stats_messages_pong_sent:10509
cluster_stats_messages_meet_sent:11
cluster_stats_messages_sent:21933
cluster_stats_messages_ping_received:10509
cluster_stats_messages_pong_received:10535
cluster_stats_messages_received:21044Copy the codeCluster_state: OK indicates that the Redis cluster is online successfully.
A master-slave replication
The Redis cluster is successfully online, but the primary node has not been assigned a secondary node. If one node fails, the whole cluster will hang and the high availability will not be achieved. Learn more about Redis master-slave replication and the principle of master-slave replication
You need to use the cluster replicate command to manually configure the primary node for the secondary node.
Cluster replication commands are as follows:
cluster replicate node-idCopy the codeEach node in the cluster node – id can be used to cluster nodes command to view, the following output 1 b4b3741945d7fed472a1324aaaa6acaa1843ccb is the master node B node – id.
[root@localhost /]# /usr/local/bin/redis-cli -p 8002 cluster nodes1 b4b3741945d7fed472a1324aaaa6acaa1843ccb 127.0.0.1:7002 @ 17002 master 1552220700255-0 1 connected Ed6fd72e61b747af3705b210c7164bc68739303e 127.0.0.1:8003 @ 18003 master - 0, 1552220703281 connected 19147 f56e679767bcebb8653262ff7f56ca072a8 127.0.0.1:7003 @ 17003 master - 0 1552220700000 2 connected a8a41694f22977fda78863bdfb3fc03 dd1fab1bd 127.0.0.1:8002 @ 18002 myself, master - 0 1552220701000 5 61 e8c4ed8d1ff2a765a4dd2c3d300d8121d26e12 connected 127.0.0.1:7001 @ 17001 master - 0 1552220702275 4 connected 51987 c4b5530c81f2845bb9d521daf6d3dce3659 127.0.0.1:8001 @ 18001 master - 0 1552220701265 3 connectedCopy the codeAccording to the planning in advance, A master D slave; B primary E secondary; C primary F secondary. Run the following commands to specify primary nodes for secondary nodes D, E, and F so that the cluster can automatically complete the primary/secondary replication.
/usr/local/bin/redis-cli -p 8001 cluster replicate 61e8c4ed8d1ff2a765a4dd2c3d300d8121d26e12
/usr/local/bin/redis-cli -p 8002 cluster replicate 1b4b3741945d7fed472a1324aaaa6acaa1843ccb
/usr/local/bin/redis-cli -p 8003 cluster replicate 19147f56e679767bcebb8653262ff7f56ca072a8Copy the codeWhen this command is executed successfully, we have set up a Redis cluster manually.
Finally, take a look at the node information in the cluster.
[root@localhost ~]# /usr/local/bin/redis-cli -p 8002 cluster nodes1 b4b3741945d7fed472a1324aaaa6acaa1843ccb 127.0.0.1:7002 @ 17002 master - 0, 1552233328337 1 connected. 5462-10922 Ed6fd72e61b747af3705b210c7164bc68739303e 127.0.0.1:8003 @ 18003 slave f56e679767bcebb8653262ff7f56ca072a8 0 19147 1552233327000 2 connected 19147 f56e679767bcebb8653262ff7f56ca072a8 127.0.0.1:7003 @ 17003 master - 0 1552233325000 2 connected 10923-16383 a8a41694f22977fda78863bdfb3fc03 dd1fab1bd 127.0.0.1:8002 @ 18002 myself, slave b4b3741945d7fed472a1324aaaa6acaa1843ccb 1 0 1552233327000 5 connected 61 e8c4ed8d1ff2a765a4dd2c3d300d8121d26e12 127.0.0.1:7001 @ 17001 master - 0 4 connected a scale of 0-5461 to 1552233327327 51987 c4b5530c81f2845bb9d521daf6d3dce3659 127.0.0.1:8001 @ 18001 slave e8c4ed8d1ff2a765a4dd2c3d300d8121d26e12 0 61 1552233326320 4 connectedCopy the codeAutomatic construction
A cluster management tool, redis-trib.rb, has been released since Redis 3.0, integrated in the SRC directory of the Redis source package. It encapsulates Redis cluster command, simple and convenient to use.
However, redis-trib.rb was developed in Ruby by redis authors, so you need to install the Ruby environment on your machine before using this tool. As the authors may be aware of this problem, Redis 5.0 has integrated the tool into Redis – CLI starting with the –cluster parameter, where the create command can be used to create clusters.
Start node
Before using the cluster management tool to build a cluster, you need to start each node. For details about how to start a node, see “Manually Create a node” – “Start a Node”.
The cluster management tool is set up
If you have versions 3. X and 4. X of Redis installed, you can build with Redis -trib.rb, but you will need to install the Ruby environment.
Start by installing the Ruby environment and other dependencies with Yum.
yum -y install ruby ruby-devel rubygems rpm-buildCopy the codeConfirm the installed version.
[root@localhost redis-cluster]# ruby -vRuby 2.0.0 p648 (2015-12-16) [x86_64 - Linux]Copy the codeRun the redis-trib.rb script to set up the cluster.
/usr/local/redis-5.0.3/ SRC /redis-trib.rb create --replicas 1 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:8001 127.0.0.1:127.0.0.1 8002:8003Copy the codeHowever, the Redis version used in this paper is 5.0.3, so I can directly use Redis -cli –cluster create command to set up, as shown below.
/usr/local/bin/redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003-1 - cluster - replicasCopy the codeThe primary node comes first, followed by the secondary node. The –cluster-replicas parameter specifies the number of secondary nodes in a primary node. For example, –cluster-replicas 1 indicates that each primary node has one secondary node.
If the command is successfully executed, the following information is displayed:
[root@localhost bin]# redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:8001 to 127.0.0.1:7001
Adding replica 127.0.0.1:8002 to 127.0.0.1:7002
Adding replica 127.0.0.1:8003 to 127.0.0.1:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 32f9819fc7 d561bfa2b7189182200e86d9901b8a 127.0.0.1:7001 slots: [0-5460] (5461 slots) master M: Cca0fbfa374bc175d481e68ee9ed13b65453e967 127.0.0.1:7002 slots: [5461-10922] (5462 slots) master M: 964 cfa1c2dcfe36b6d3c63637f0d57ccb568354e 127.0.0.1:7003 slots: [10923-16383] (5461 slots) master S: 1 b47b9e6e7a79523579b8d2ddcd5e708583ed317 127.0.0.1:8001 32 f9819 replicatesfc7d561bfa2b7189182200e86d9901b8a S: Aba9330f3e70f26a8af4ced1b672fbcc7bc62d78 127.0.0.1:8002 replicates cca0fbfa374bc175d481e68ee9ed13b65453e967 S: 254 db0830cd764 e075aa793144572d5fa3a398f0 127.0.0.1:8003 replicates 964 cfa1c2dcfe36b6d3c63637f0d57ccb568354e Can Iset the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: 32f9819fc7 d561bfa2b7189182200e86d9901b8a 127.0.0.1:7001 slots: [0-5460] (5461 slots) master 1 additional up (s) s: Aba9330f3e70f26a8af4ced1b672fbcc7bc62d78 127.0.0.1:8002 slots: (0 slots) slave replicates cca0fbfa374bc175d481e68ee9ed13b65453e967 S: 1 b47b9e6e7a79523579b8d2ddcd5e708583ed317 127.0.0.1:8001 slots: (0 slots) slave 32 f9819 replicatesfc7d561bfa2b7189182200e86d9901b8a
S: 254db0830cd764 e075aa793144572d5fa3a398f0 127.0.0.1:8003 slots: (0 slots) slave replicates 964cfa1c2dcfe36b6d3c63637f0d57ccb568354e M: Cca0fbfa374bc175d481e68ee9ed13b65453e967 127.0.0.1:7002 slots: [5461-10922] (5462 slots) master 1 additional up (s) M: 964 cfa1c2dcfe36b6d3c63637f0d57ccb568354e 127.0.0.1:7003 slots: [10923-16383] (5461 slots) master 1 additional up (s) [OK] All nodes agree about slots configuration. >>> Checkfor open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.Copy the codeOK, setup complete! One command.
END
Author: Esofar
cnblogs.com/esofar/p/10486621.html