Why does Redis want master-slave replication?

  • Read and write separation, improve profane performance
  • Data backup reduces the risk of data loss
  • High availability, avoiding single point of failure

How to copy?

The general idea is to take the master server data as the standard, and synchronize all data when connecting the master server to the slave server. Then, if the master server executes a command, the master server synchronizes the command to the slave server. This is what we call the separation of writing from reading. In fact, this was the case before Redis 2.8.

Redis full copy

Timing: Occurs during slave server initialization

The master server receives the SYNC command, generates an RBB file (a bytecode file), and uses the cache to record all write commands executed thereafter. This purpose is to save the RBD file generated by the incremental command from the master server to the slave server after the RDB file is generated. The RBD file is loaded from the master server to the slave server, and the slave server executes the incremental write command

Redis incremental synchronization:

Timing: Write operations that occur on the master server after the slave server is initialized

Incremental replication is when a write command is executed by the master server and the same command is received and executed by the master server.

Note: master/slave replication is asynchronous, does not block master/slave redis threads (this is very important, because redis is a single thread) master server can still process external commands, slave server can also accept external queries, but to check the old data.

How does Redis handle key expiration in master-slave mode?

The master server does not allow a key to expire. When a master server deletes an expired key, it generates a del command for all slave servers

int expireIfNeeded(redisDb *db, robj *key) { time_t when = getExpire(db,key); if (when < 0) return 0; /* No expire for this key */ /* Don't expire anything while loading. It will be done later. */ if (server.loading) return 0; // If (server.masterhost! = NULL) { return time(NULL) > when; } /* Return when this key has not expired */ if (time(NULL) <= when) return 0; /* Delete the key */ server.stat_expiredkeys++; propagateExpire(db,key); return dbDelete(db,key); }Copy the code

Problem: If the slave server is constantly reconnecting, the master server is constantly generating full data

Redis 2.8:

General idea:

  • The server maintains an offset and modifies the offset when there is a write command.
  • The master server maintains a fixed-size buffer (configurable) to write commands to when received
  • The slave server also maintains an offset and modifies it when receiving a master server write command.
  • When connecting from a server to a master server, an offset and a master SERVER ID (possibly multi-master) are sent to the master server.

Follow the operation process

  1. If the active server does not match the active server, the active server is fully synchronized
  2. The offset is checked to see if it is buffered (it may be behind, and the cache has been overwritten), and full synchronization is performed
  3. If the offset is found in the buffer, the master follows the offset and writes the buffer to the slave server

Note: the size of the main server cache becomes very important, the larger it is, the closer it is to full update and the smaller it is to full update when the current network is bad.

The synchronization code is as follows:
/* replication. C: line 627 */ void syncCommand(client *c) {// If client is a server, ignore if (c->flags & CLIENT_SLAVE) return; If (server.masterhost && server.repl_state!) if (server.masterhost && server.repl_state! = REPL_STATE_CONNECTED) { addReplySds(c,sdsnew("-NOMASTERLINK Can't SYNC while not connected with my master\r\n")); return; } // If there is any data in the client buffer, If (clientHasPendingReplies(c)) {addReplyError(c,"SYNC and PSYNC are invalid with pending output"); return; ServerLog (LL_NOTICE,"Slave %s asks for synchronization", replicationGetSlaveName(c)); // If psync if (! Strcasecmp (c->argv[0]-> PTR,"psync") { If success proves don't need the whole quantity of the if (masterTryPartialResynchronization (c) = = C_OK) {/ / can perform PSYNC command, Server.stat_sync_partial_ok ++; server.stat_sync_partial_ok++; // Complete synchronization is not required. Char *master_replid = c->argv[1]-> PTR; char *master_replid = c->argv[1]-> PTR; // From the server? If (master_replid[0]! +1 if (master_replid[0]! = '? ') server.stat_sync_partial_err++; }} else {/ / sync command c - > flags | = CLIENT_PRE_PSYNC; } // All synchronization statistics +1 server.stat_sync_full++; C -> replState = SLAVE_STATE_WAIT_BGSAVE_START; // Whether to disable TCP_NODELAY if (server.repl_DISABle_tcp_nodelay) // Enable the Nagle algorithm anetDisableTcpNoDelay(NULL, c->fd); // Save the RDB file fd from the master service and set it to -1 c->repldbfd = -1; / / logo for a client from the server c - > flags | = CLIENT_SLAVE; ListAddNodeTail (Server.slaves,c); // Add client to the slave server list. If there is only one element from the server list and the backlog is empty, If (listLength(server.slaves) == 1 && server.repl_backlog == NULL) {// Initialize replication ID, ChangeReplicationId () is used to assign a unique ID to the subsequent master-slave mode. // Clear the secondary replication ID, which may occur when a new replication history is started after a full resynchronization; clearReplicationId2(); // Initialize backlog createReplicationBacklog(); } // Case 1: BGSAVE is already being executed and synchronized to disk if (server.rdb_child_pid! = -1 && server.rdb_child_type == RDB_CHILD_TYPE_DISK) { client *slave; listNode *ln; listIter li; listRewind(server.slaves,&li); While ((ln = listNext(&li))) {slave = ln->value; If (slave-> replState == SLAVE_STATE_WAIT_BGSAVE_END) break; if (slave-> replState == SLAVE_STATE_WAIT_BGSAVE_END) } // For this slave server, we check if it has the ability to trigger the current BGSAVE operation, If ((c->slave_capa & slave->slave_capa) == slave->slave_capa)) {// CopyClientOutputBuffer (C,slave) copy all output buffer contents of slave to all output buffer contents of C; / / set the amount of heavy state of sync from the server, set up the part weight of synchronization offset replicationSetupSlaveForFullResync (c, slave - > psync_initial_offset); serverLog(LL_NOTICE,"Waiting for end of BGSAVE for SYNC"); } else {serverLog(LL_NOTICE,"Can't attach the slave to the current bgsave. Waiting for next BGSAVE for SYNC"); } else if (server.rdb_child_pid! Rdb_child_type == RDB_CHILD_TYPE_SOCKET) {rdb_child_type == RDB_CHILD_TYPE_SOCKET) { BGSAVE serverLog(LL_NOTICE,"Current BGSAVE has socket target. Waiting for next BGSAVE for SYNC"); // Case 3: BGSAVE not executing} else {// If the server supports diskless synchronization if (server.repl_diskLESS_sync && (c->slave_capa & SLAVE_CAPA_EOF)) {// The child process of diskless synchronous replication is created in replicationCron() and delayed because it wants to wait for more slave servers to arrive. If (server. repl_diskLESS_sync_delay) serverLog(LL_NOTICE,"Delay next BGSAVE for diskless SYNC"); } else {// Server does not support diskless replication // If no BGSAVE is being executed and no AOF file is being written, start BGSAVE for replication, And RDB file to disk if (for server aof_child_pid = = 1) {startBgsaveForReplication (c - > slave_capa); } else { serverLog(LL_NOTICE, "No BGSAVE in progress, but an AOF rewrite is active. BGSAVE for replication delayed"); } } } return; }Copy the code

The following will be in the form of this note to analyze Redis cluster ~

This article is basically an old suture in the form of notes strange, others analyze too well. I’ll just take it and use it. To delete (cut)

Reference article:

www.web-lovers.com/redis-sourc…

Github.com/farmerjohng…