Redis-cluster mode cluster construction | August more article challenge

This is the 12th day of my participation in the August Text Challenge.More challenges in August

First, build a cluster

Redis Cluster requires three masters to form a Cluster. Therefore, I use the mode of three masters and three slaves here. In actual production, there are at least three masters and six slaves in high availability

Conf configuration file must be cluster-enabled yes Enable cluster mode, use docker-compose to start six Redis containers, and use host network. Otherwise, jedis internal REdirection IP addresses are incorrect.

version: '3.7'
services:
  redis7001:
    image: 'redis: 6.2.4'
    container_name: redis7001
    network_mode: "host"
    command:
      ["redis-server"."/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7001/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7001/data:/data
    ports:
      - "7001:7001"
      - "17001:17001"
    environment:
      - TZ=Asia/Shanghai
  redis7002:
    image: 'redis: 6.2.4'
    container_name: redis7002
    network_mode: "host"
    command:
      ["redis-server"."/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7002/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7002/data:/data
    ports:
      - "7002:7002"
      - "17002:17002"
    environment:
      - TZ=Asia/Shanghai
  redis7003:
    image: 'redis: 6.2.4'
    container_name: redis7003
    network_mode: "host"
    command:
      ["redis-server"."/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7003/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7003/data:/data
    ports:
      - "7003:7003"
      - "17003:17003"
    environment:
      - TZ=Asia/Shanghai
  redis7004:
    image: 'redis: 6.2.4'
    container_name: redis7004
    network_mode: "host"
    command:
      ["redis-server"."/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7004/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7004/data:/data
    ports:
      - "7004:7004"
      - "17004:17004"
    environment:
      - TZ=Asia/Shanghai 
  redis7005:
    image: 'redis: 6.2.4'
    container_name: redis7005
    network_mode: "host"
    command:
      ["redis-server"."/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7005/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7005/data:/data
    ports:
      - "7005:7005"
      - "17005:17005"
    environment:
      - TZ=Asia/Shanghai 
  redis7006:
    image: 'redis: 6.2.4'
    container_name: redis7006
    network_mode: "host"
    command:
      ["redis-server"."/etc/redis/redis.conf"]
    volumes:
      - /opt/docker/redis/7006/redis.conf:/etc/redis/redis.conf
      - /opt/docker/redis/7006/data:/data
    ports:
      - "7006:7006"
      - "17006:17006"
    environment:
      - TZ=Asia/Shanghai 
Copy the code

Check whether the Redis container is started successfully.

[root@shang redis]# docker ps
CONTAINER ID   IMAGE         COMMAND                  CREATED          STATUS         PORTS                                                                                                NAMES
e3018a083b37   redis:6.2.4   "Docker - entrypoint. S..."   13 seconds ago   Up 4 seconds   0.0.0.0:7003->7003/tcp, :::7003->7003/tcp, 6379/tcp, 0.0.0.0:17003->17003/tcp, :::17003->17003/tcp   redis7003
911dc36fe35a   redis:6.2.4   "Docker - entrypoint. S..."   13 seconds ago   Up 4 seconds   0.0.0.0:7005->7005/tcp, :::7005->7005/tcp, 6379/tcp, 0.0.0.0:17005->17005/tcp, :::17005->17005/tcp   redis7005
efabccd7769c   redis:6.2.4   "Docker - entrypoint. S..."   13 seconds ago   Up 6 seconds   0.0.0.0:7004->7004/tcp, :::7004->7004/tcp, 6379/tcp, 0.0.0.0:17004->17004/tcp, :::17004->17004/tcp   redis7004
db48c12796cb   redis:6.2.4   "Docker - entrypoint. S..."   13 seconds ago   Up 4 seconds   0.0.0.0:7002->7002/tcp, :::7002->7002/tcp, 6379/tcp, 0.0.0.0:17002->17002/tcp, :::17002->17002/tcp   redis7002
c717621d2c7b   redis:6.2.4   "Docker - entrypoint. S..."   13 seconds ago   Up 4 seconds   0.0.0.0:7006->7006/tcp, :::7006->7006/tcp, 6379/tcp, 0.0.0.0:17006->17006/tcp, :::17006->17006/tcp   redis7006
10fd3e24f10f   redis:6.2.4   "Docker - entrypoint. S..."   13 seconds ago   Up 4 seconds   0.0.0.0:7001->7001/tcp, :::7001->7001/tcp, 6379/tcp, 0.0.0.0:17001->17001/tcp, :::17001->17001/tcp   redis7001
Copy the code

The Redis cluster is shown as a cluster, but there is no connection between them and no slot assigned.

[root@shang 7001]# ps -ef | grep redis
polkitd    3888   3867  0 09:10 ?        00:00:02 redis-server* :7001 [cluster]
polkitd    4143   4120  0 09:14 ?        00:00:01 redis-server* :7002 [cluster]
polkitd    4257   4235  0 09:17 ?        00:00:01 redis-server* :7003 [cluster]
polkitd    4373   4352  0 09:20 ?        00:00:00 redis-server* :7004 [cluster]
polkitd    4490   4469  0 09:22 ?        00:00:00 redis-server* :7005 [cluster]
polkitd    4604   4581  0 09:23 ?        00:00:00 redis-server* :7006 [cluster]
root       4667   1576  0 09:24 pts/0    00:00:00 grep --color=auto redis

[root@shang 7001]# docker exec -it redis7001 redis-cli -p 7001
127.0.0.1:7001> cluster nodes # Query cluster node information
87346232e7666c772d304518fa4215804be3f576 :7001@17001 myself,master - 0 0 0 connected
Copy the code

Establish connections and assign slots.

# View the help command
[root@shang 7001]# docker exec -it redis7001 redis-cli --cluster help 
# meet
create  host1:port1 ... hostN:portN
          --cluster-replicas <arg>
Copy the code

Docker exec it redis7001 redis-cli –cluster create 192.168.16.3:7001 192.168.16.3:7002 192.168.16.3:7003 192.168.16.3:7004 192.168.16.3:7005 192.168.16.3:7006 –cluster-replicas 1 Establishes cluster relationships based on 1:1 (3:3). The first three replicas are the masters. The last three are slaves. If –cluster-replicas 2, it indicates that the first three are masters and the last three are slaves in the 1:2 (3:6) manner

Check whether the establishment is successful

[root@shang 7001]# docker exec -it redis7001 redis-cli -p 7001
127.0.0.1:7001> cluster nodes # Query cluster node information
Copy the code

2. Cluster expansion procedure

docker exec -it redis7001 redis-cli --cluster help # View the help command
add-node  new_host:new_port existing_host:existing_port # Any redis in the new Redis cluster
          --cluster-slave  # flag for slave machine
          --cluster-master-id <arg> Master-id specifies the master host
# Add a slave
docker exec -it redis7001 redis-cli --cluster add-node 192.168.16.3:7007 192.168.16.3:7001 --cluster-slave --cluster-master-id 87346232e7666c772d304518fa4215804be3f576

# add a master and allocate slot
docker exec -it redis7001 redis-cli --cluster add-node 192.168.16.3:7008 192.168.16.3:7001

Copy the code

reshard  host:port # Any node in the cluster
         --cluster-from <arg> # Id of the master node with slot, separated by commas
         --cluster-to <arg> Which master node ID is assigned to
         --cluster-slots <arg> # How many slots are allocated
         --cluster-yes # skip confirmation
         --cluster-timeout <arg> # 
         --cluster-pipeline <arg> # control the number of keys in a batch migration. Default is 10
         --cluster-replace
		
docker exec -it redis7001 redis-cli --cluster reshard 192.168.16.3:7007 --cluster-from 87346232e7666c772d304518fa4215804be3f576,4979d1b009f30576c4ba7587c01176275cf81628 --cluster-to 90afcac7725cac92e9b064f3a71348b9913c7eb3  --cluster-slots 1000 --cluster-yes
Copy the code

3. Cluster capacity reduction procedure

1. Remove all slaves of a master node to prevent the master node from being elected.
2. Before removing a master, allocate slots on the master to other active masters and then delete the current master to avoid data loss.

# remove slave
 del-node host:port node_id # Id of any cluster node to be deleted
 
 docker exec -it redis7001 redis-cli --cluster del-node 192.168.16.3:7001 3a29bd1d4e885cdfe5e03185183b88da2cfd5115
# separation slot
docker exec -it redis7001 redis-cli --cluster reshard 192.168.16.3:7001 --cluster-from 90afcac7725cac92e9b064f3a71348b9913c7eb3 --cluster-to 05abce1703ec1aca5e38548ca2c252c4713c5840 --cluster-slots 1000 --cluster-yes
Copy the code

 docker exec -it redis7001 redis-cli --cluster del-node 192.168.16.3:7001 90afcac7725cac92e9b064f3a71348b9913c7eb3 Remove master 7008
Copy the code

4. Record points

Due to the different networks of the host machine (192.168.xx.xx) and the Docker container (170.17.0.x), the Redis cluster operated by Jedis cannot obtain connections. The root cause is that the HostAndPort in the makeObject in JedisFactory was changed to the Docker network instead of the external IP of our host machine, so the connection could not be created and the fetch failed.