This is the fourth day of my participation in Gwen Challenge

preface

The storage capacity of Redis on a single machine is limited by the single machine. Therefore, load balancing and read/write separation functions cannot be implemented to ensure high availability. Therefore, the master-slave architecture of Redis should be built to avoid these problems

Master-slave replication refers to the replication of data from one Redis server to other Redis servers. Master-slave is the basis for sentry and cluster mode to be implemented.

Master/slave replication

  • Data redundancy: Master/slave replication implements hot backup of data, which is a kind of data redundancy in addition to persistence.
  • Fault recovery: When the primary node is faulty, the secondary node provides services for rapid fault recovery.
  • Load balancing: On the basis of master/slave replication and read/write separation, the master node provides the write service and the slave node provides the read service to share server load. Especially in the scenario of less write and more read, the concurrency of the Redis server can be greatly increased by sharing the read load with multiple slave nodes.
  • Read and write separation: the master library writes and the slave library reads. Read and write separation can not only improve the load capacity of the server, but also change the number of slave libraries according to the change of demand.
  • High availability cornerstone: In addition to the above, master-slave replication is the foundation upon which sentry and clustering can be implemented.

Since the role of master-slave architecture is so powerful, this article will talk to you about the broken things of Redis master-slave architecture, which can be divided into the following four parts:

  • A master-slave architectureDetailed explanation of the construction process
  • A master-slave architectureData synchronization process
  • How about Java codeConnect to the Redis master-slave architecture on the remote cloud server
  • Extra curricular tips – How to checkRedis service running information

Let’s start with the first part of the introduction: master/slave architecture construction process in detail

Part 1: Master/slave architecture establishment process

  • Step 1: Modify the configuration file of the primary node
# Change the protection mode to noprotected-mod no # bind the local network card address, ali cloud server to bind the internal NETWORK IP, plus127.0. 01.Redis -cli -h xxx.xxx.XXX bind xxx.xx.xx.xxx127.0. 01.Change the port of the master node7000Write the pid process number to pidfile /var/run/redis_7000.pid # logfile name logfile"redis_7000.log"Dir/XXX/XXX /Copy the code
  • Step 2: Start the primary node

  • Step 3: Create a folderIt is used to store the configuration file of the secondary node

  • Step 4:Make a copy of the redis.conf fileTo the newly created directory as a configuration file for the slave node,Example Change the name of a configuration fileEasy to identify,Example Modify the configuration file of the slave node
# Change the protection mode to noprotected-mod no # Bind the network adapter address of the local server to the internal IP address of the Ali cloud server8888Write the pid process number to pidfile /var/run/redis_8888. Pid # logfile name logfile"redis_8888.log"The absolute path is dir/XXX/XXX /slave/01/ # # configuration of master-slave replication master node network card address, ali, there is a binding cloud server network IP replicaof master node IP port # master node configuration from the node is read-only up - read - only yesCopy the code
  • Step 5: Start the slave node

Test whether the build is successful

  • Used on the client side of the slave nodekeys *Command to find that data has been fully synchronized on the primary node

  • Enter theClient Settings of the primary nodeAnd then inFetch from nodeThe data has been synchronized

  • Enter the client of the primary nodeinfo replicationCommand to check the connection status

  • Go to the client of the slave node and enterinfo replicationCommand to check the connection status, take one of the secondary nodes as an example, which has the connection status of the primary node:upThe connection is successful.

This proves that the master-slave architecture environment of Redis has been set up successfully

The pit that the master stepped on during the construction process

  • Make sure the file directory exists, like the one below, toMake sure it exists before starting the slave node, otherwise,The slave node will not start up

  • The bound IP address is not modifiedData synchronization failedSee the following article for reasons:Redis’ bind errorThe solution isBind the IP address of the local NIC, or simply set it to unlimited

Part 2: Redis master/slave synchronization process

The synchronization of the Redis slave and master nodes is divided into two scenarios:

  1. The first scenario: boot from the master node at slave node timeFull amount of synchronization
  2. Scenario 2: The master node has delta commands that are synchronized to the slave node,The incremental synchronization

Full amount of synchronization

The flow chart

Process analytical

  • Step 1: The slave node is first established with the master nodeThe socket length linkKeep up correspondence
  • Step 2: Send from the nodepsyncCommand to initiate a request for synchronization
  • Step 3: The primary node receives the psync command and executesBgsave command, generate an RDB snapshot, and willIncremental commandStored in a buffer
  • Step 4: After the primary node generates a fast RDB snapshot, it sends the binary data of the RDB snapshot to the secondary node
  • Step 5: Receive the RDB snapshot data from the primary node, clear the old data, and load it into the memory
  • Step 6: Wait until the secondary node finishes synchronizing the snapshot data, and the primary node sends the snapshot dataIncremental commandTo the slave node
  • Step 7: Execute the delta command from the node
  • Step 8: Master nodeLong connection through socketContinuing theThe commandSend to the slave node

Matters needing attention

When the master node receives a concurrent connection request from multiple secondary nodes, it persists only once and sends this persistent data to multiple secondary nodes that are concurrently connected

The incremental synchronization

The flow chart

Process analytical

  • Step 1: Disconnect the slave node from the master nodeThe socket length link
  • Step 2: Re-send the socket connection request from the node to establish communication
  • Step 3: Send from the nodepsync(offset)Command to initiate a request for synchronization
  • Step 4: The master node checks whether offset is in the buffer of the nearest commandrepl backlog buffer, if it exists, then fromCommands after offset are synchronized to the slave onceIf no, it indicates that the disconnection takes a long time and full synchronization is performed
  • Step 5: Master nodeLong connection through socketContinuing theThe commandSend to the slave node

Precautions during master/slave synchronization

Buffer setting size

The buffer length is fixed and limited, so the write commands that can be backed up are limited. If the offset difference between the primary node and the secondary node exceeds the buffer length, partial replication cannot be performed, only full replication can be performed.

To increase the probability of partial replication in the event of a network outage, the size of the replication backlog buffer can be increased as needed (by configuring the repl-backlog-size).

For example, if the average time of network outages is 60 seconds and the average number of bytes of write commands (protocol-specific format) generated by the primary node is 100KB per second, the average requirement for replication backlogs is 6MB. To be safe, you can set it to 12MB to ensure that partial replication can be used for most disconnections.

Master/slave replication storm

If there are many secondary nodes, to alleviate the master/slave replication storm (excessive pressure on the primary node caused by multiple secondary nodes simultaneously replicating the primary node), you can use the following architecture to synchronize data between some secondary nodes and the primary nodes

Part 3: Java connection Redis master-slave architecture on remote cloud server

Configuring firewall Ports

For security groups, configure rules on the ECS interface. Lightweight servers configure port ranges on the firewall

Introduce the Jedis client

        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <version>2.9. 0</version>
        </dependency>
Copy the code

Write connection code

public class JedisSingleTest {

    public static void main(String[] args) {

        JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
        jedisPoolConfig.setMaxIdle(10);
        jedisPoolConfig.setMaxTotal(20);
        jedisPoolConfig.setMinIdle(5);
        // timeout: this is both a connectionTimeout and a read/write timeout. Starting with Jedis 2.8, the constructors for connectionTimeout and soTimeout are distinguished
        JedisPool jedisPool = new JedisPool(jedisPoolConfig, "Ali Cloud server public IP address".7000.3000.null);
        // Take a connection from the Redis connection pool and execute the command
        Jedis jedis = jedisPool.getResource();
        // Jedis common command 172.19.43.91
        System.out.println(jedis.set("single2"."aisiaoteman"));
        System.out.println(jedis.get("single2")); }}Copy the code

Test connection code

After running the main method, the console output is correct, and the master and slave nodes output is correct

Prove that Java connection remote Aliyun Redis configuration is successful

Connect remote Ali cloud Redis stepped on the pit

Configure firewall ports or ECS security groups ！！！！！

For security groups, configure rules on the ECS interface. Lightweight servers configure port ranges on the firewall

、

Ensure that the public IP address of the cloud server is used for connection, not the internal IP address ！！！！！

Part 4: Extracurricular knowledge – check the operation information of Redis service

Remember the info replication command from the previous article to check master/slave configuration information?

The info command is specifically used to view the operating information of the Redis service. The operating information of the Redis service can be divided into nine chunks:

  • Server: environment parameter of the server running

  • Clients: indicates information about the client

  • Memory: indicates the statistics of server running memory

  • Persistence: Persisting information

  • Stats: General statistics

  • Replication: Information about primary and secondary replication

  • CPU: information about the CPU

  • Cluster: indicates the cluster information

  • KeySpace: indicates the statistics of key/value pairs

You can view the information directly in the info module name. For example, to view master/slave replication information, type info replication

conclusion

Ok, that is all the content of this article, this article from four aspects to give you a more intuitive Redis master-slave architecture:

  • A master-slave architectureDetailed explanation of the construction process
  • A master-slave architectureData synchronization process
  • How about Java codeConnect to the Redis master-slave architecture on the remote cloud server
  • Extra curricular tips – How to checkRedis service running information

We believe that now everyone is now a master slave Redis architecture, but this is not the end, we in the next article in order to achieve the high availability of Redis master slave, but also based on the master slave architecture, continue to build the Redis sentinel cluster, to achieve automatic fault recovery, you can look forward to ！！！！

Phase to recommend

Starting from scratch to learn Redis series (a) | Redis environment set up

Starting from scratch to learn Redis series (2) | how Redis commands commonly used jolt in your business

From scratch learn Redis series (3) | how to correct the fencing Redis persistence

omg

Finally, if you feel confused about the article, please leave a comment immediately. If you think I have something, please like 👍, follow ❤️ and share 👥, because this will be my motivation to output more high-quality articles, thank you!!

If you want to get books related to Redis, you can follow the wechat official account Java Encyclopedia and enter Redis

Finally, thank you for your support. See you next time!