IT brother

A big factory to do advanced Java development program ape

❝

Follow wechat public account: IT elder brother

❞

❝

Java actual combat project video tutorial: you can get 200G, 27 sets of actual combat project video tutorial

❞

❝

Reply: Java learning route, you can get the latest and most complete a learning roadmap

❞

❝

Re: Java ebooks, get 13 must-read books for top programmers

❞

❝

Java foundation, Java Web, JavaEE all tutorials, including Spring Boot, etc

❞

❝

Reply: Resume template, you can get 100 beautiful resumes

❞

preface

Before the elder brother talked about Redis common command daqo, Redis persistent backup and disaster recovery processing. Today we will talk about how master slave replication is implemented in Redis.


Redis primary/secondary replication

concept

The concept of Redis master-slave replication is roughly similar to that of MySQL. One master host and one slaver slave host. Master Data is automatically synchronized to the slaver Slave machine based on the configuration and policies. Master data is mainly written and Slave data is mainly read.


The main purpose

  • Read/write separation: It is suitable for applications with more read and less write. Multiple slave machines are added to improve read speed and program concurrency

  • Data Dr: Replicates data on the host from the slave host. If data on the host is lost, data stored on the slave host can be used to restore data.

  • High concurrency and HIGH availability cluster implementation basis: In a high concurrency scenario, even if the host hangs up, the slave host can perform the primary/secondary switchover and automatically become the host to provide external services.

One master multiple slave configuration


Environment to prepare

My brother is so poor that he uses one machine to simulate three machines.

  • Step 1: make three copies of redis.conf: redis6379.conf, redis6380.conf, redis6381.conf

  • Step 2: Modify the port name, PID name, log name and RDB name in the three redis

  • Step 3: open three Windows respectively to simulate three servers and start redis service.

View the primary and secondary roles of the current three machines

Start by using the command Info replication to see what roles the three machines are currently in.

All three machines are in this state

127.0.0.1:6379 > info replication

# The role is master

role:master

The number of slave machines is 0

connected_slaves:0

Copy the code

Set the primary/secondary relationship

Note here that we only set the slave machine, not the host. We choose 6380 and 6381 as slave machines. 6379 as the host.

# 6380 port

127.0.0.1:6380 > SLAVEOF 127.0.0.1 6379



# 6381 port

127.0.0.1:6381 > SLAVEOF 127.0.0.1 6379

Copy the code

Check the current roles of the three machines again

Run info replication again

# host

127.0.0.1:6379 > info replication

role:master # Role: host

connected_slaves:2 # number of connected slave machines, as well as the IP and port of slave machines

Slave0: IP = 127.0.0.1 port = 6380, state = online, offset = 98, lag = 1

Slave1: IP = 127.0.0.1 port = 6381, state = online, offset = 98, lag = 1



# 1 from the machine

127.0.0.1:6380 > info replication

role:slave # Role: Slave machine

Master_host: 127.0.0.1The IP address and port of the host

master_port:6379



# 2 from the machine

127.0.0.1:6381 > info replication

role:slave # Role: Slave machine

Master_host: 127.0.0.1The IP address and port of the host

master_port:6379

Copy the code

Build it. Try it out

  • Full replication: The slave machine synchronizes all the previous data from the host. You can try to get a key on the slave machine.

  • Incremental replication: When new data is added to the host, you can run the set key value command on the host and get the key on the slave to check whether the new data can be obtained.

Reading and writing separation

The slave machine of Redis does not allow write operations by default. If you run the set key value command on the slave machine, an error will be reported.

# 6380 from the machine

127.0.0.1:6380 >set k3 v3

(error) READONLY You can't write against a read only slave.

Copy the code

“Shout, very tired”, master copy write of almost!!


Principle of master-slave replication


Full amount of copy


“①” The slave sends psync. Since it is the first replication, it does not know the master’s Runid and offset, so it sends psync? – 1

② The master receives the request and sends the master’s runid and offset to the slave node.

③ saves the master information from the slave node

④ The active node bgSave saves RDB files

⑤ The host point sends the RDB file

The data generated during “④” and “⑤” will be written to the replication buffer repl_back_buffer.

“⑥” The master node sends the buffer generated in the previous two steps to the slave node

“⑦” clears the original data from the node. If it had data before, it will clear the data after a long time

⑧ loads the RDB file data from the slave node into itself.

The cost of full replication

“①” BGSave time

“②” RDB file network transmission time

③ Clears data from the node

④ Time to load the RDB from the node

“⑤” possible AOF rewriting time, which is specific to the slave node. For example, after AOF is enabled, AOF rewriting may be required when adding buffer data from the node

Due to the above reasons, some cases are not suitable for full replication. For example, after network jitter, only part of the data needs to be transmitted from the secondary node, but not all of the data needs to be transmitted. Partial replication is implemented after Redis2.8

Part of the copy


Assume that the sending network jitter or other conditions, and the connection is temporarily lost

At this point, the master is still writing data to the buffer

The slave is connected to the master again

④ The slave sends its offset and RUNID to the master

“⑤” The master checks whether the slave’s offset is in the buffer queue. If so, continue to the slave. Otherwise, full copy is required (because this means that too much data has been missed).

“⑥” The master sends data to the slave from the offset to the end of the buffer queue