0 x00 the
Be careful before using a new feature. In addition to doing a lot of testing, if you can, read the code to see how it works internally.
In this article, we will look at the Redis SwapDB command through the source code.
0 x01 SWAPDB basis
1.1 Command Description
Available version: >=4.0.0
This command can exchange two databases on the same Redis server, so that the connection of one DATABASE can immediately access the data of other databases.
After swAPDB is executed, users do not need to perform the select operation to connect to the DB and can see the new data.
1.2 presentation
redis> set mystring 0 Set db 0 to 0 first
OK
redis> select 1 Then switch to DB 1
OK
redis[1] >set mystring 1 # set to 1
OK
redis[1]> swapdb 0 1 Exchange db0 and db1 data
OK
redis[1]> get mystring Db1: retrieve data from db0
"0"
Copy the codeLet’s take a look at the source code, what Redis actually doing behind the scenes, and whether this feature has an impact on our daily business.
0x02 Pre-check
The SWAPDB entry function is swapdbCommand.
As you can see, swapdbCommand does some checking up front.
- If the cluster mode is used, the switchover is not allowed.
- Get two DB idNexes, if error, do not switch;
Then we call dbSwapDatabases to switch;
/* SWAPDB db1 db2 */
void swapdbCommand(client *c) {
long id1, id2;
/* Not allowed in cluster mode: we have just DB 0 there. */
if (server.cluster_enabled) {
addReplyError(c,"SWAPDB is not allowed in cluster mode");
return;
}
/* Get the two DBs indexes. */
if (getLongFromObjectOrReply(c, c->argv[1], &id1,
"invalid first DB index") != C_OK)
return;
if (getLongFromObjectOrReply(c, c->argv[2], &id2,
"invalid second DB index") != C_OK)
return;
/* Swap... * /
if (dbSwapDatabases(id1,id2) == C_ERR) {
addReplyError(c,"DB index is out of range");
return;
} else {
RedisModuleSwapDbInfo si = {REDISMODULE_SWAPDBINFO_VERSION,id1,id2};
moduleFireServerEvent(REDISMODULE_EVENT_SWAPDB,0,&si);
server.dirty++;
addReply(c,shared.ok); }}Copy the code0x03 Formal Switchover
DbSwapDatabases is a formal service process.
Look at the first half of the code, really did not expect so simple, is simple db1, DB2 some variables do exchange!
After looking at the second half of the code, it turns out that it is still a bit complicated and has some impact on the business, specifically:
- Notify redis that all clients connected to the db are ready, because some clients are listening for data using B[LR]POP, and some values may already be ready after exchanging databases.
- Notify the client of Watch above Redis DB that there is a problem with the data of this database, so the client needs to deal with it;
Details are as follows:
int dbSwapDatabases(long id1, long id2) {
if (id1 < 0 || id1 >= server.dbnum ||
id2 < 0 || id2 >= server.dbnum) return C_ERR;
if (id1 == id2) return C_OK;
redisDb aux = server.db[id1];
redisDb *db1 = &server.db[id1], *db2 = &server.db[id2];
/* Swap hash tables. Note that we don't swap blocking_keys, * ready_keys and watched_keys, since we want clients to * remain in the same DB they were. */
db1->dict = db2->dict;
db1->expires = db2->expires;
db1->avg_ttl = db2->avg_ttl;
db1->expires_cursor = db2->expires_cursor;
db2->dict = aux.dict;
db2->expires = aux.expires;
db2->avg_ttl = aux.avg_ttl;
db2->expires_cursor = aux.expires_cursor;
/* Now we need to handle clients blocked on lists: as an effect * of swapping the two DBs, a client that was waiting for list * X in a given DB, may now actually be unblocked if X happens * to exist in the new version of the DB, after the swap. * * However normally we only do this check for efficiency reasons * in dbAdd() when a list is created. So here we need to rescan * the list of clients blocked on lists and signal lists as ready * if needed. * * Also the swapdb should make transaction fail if there is any * client watching keys */
scanDatabaseForReadyLists(db1);
touchAllWatchedKeysInDb(db1, db2);
scanDatabaseForReadyLists(db2);
touchAllWatchedKeysInDb(db2, db1);
return C_OK;
}
Copy the code3.1 Notifying the Client of Ready
Because some clients are listening for data using B[LR]POP, some values may already be ready by exchanging databases.
Therefore, the first thing to do is to notify the clients of the two databases, that is, to iterate over the list of keys in the database and try to get the corresponding value. If the value can be obtained, the client will be notified that the key is ready.
/* Helper function for dbSwapDatabases(): scans the list of keys that have * one or more blocked clients for B[LR]POP or other blocking commands * and signal the keys as ready if they are of the right type. See the comment * where the function is used for more info. */
void scanDatabaseForReadyLists(redisDb *db) {
dictEntry *de;
dictIterator *di = dictGetSafeIterator(db->blocking_keys);
while((de = dictNext(di)) ! =NULL) {
robj *key = dictGetKey(de);
robj *value = lookupKey(db,key,LOOKUP_NOTOUCH);
if (value) signalKeyAsReady(db, key, value->type);
}
dictReleaseIterator(di);
}
Copy the code3.2 Notifying the Watch Client
This is to inform the client of Watch that there is a problem with the data of this database, so the client needs to deal with it.
As you can see, watched Keys are iterated over, the clients corresponding to these keys are obtained, and the flags of these clients are added to CLIENT_DIRTY_CAS.
/* Set CLIENT_DIRTY_CAS to all clients of DB when DB is dirty. * It may happen in the following situations: * FLUSHDB, FLUSHALL, SWAPDB * * replaced_with: for SWAPDB, the WATCH should be invalidated if * the key exists in either of them, and skipped only if it * doesn't exist in both. */
void touchAllWatchedKeysInDb(redisDb *emptied, redisDb *replaced_with) {
listIter li;
listNode *ln;
dictEntry *de;
if (dictSize(emptied->watched_keys) == 0) return;
dictIterator *di = dictGetSafeIterator(emptied->watched_keys);
while((de = dictNext(di)) ! =NULL) {
robj *key = dictGetKey(de);
list *clients = dictGetVal(de);
if(! clients)continue;
listRewind(clients,&li);
while((ln = listNext(&li))) {
client *c = listNodeValue(ln);
if (dictFind(emptied->dict, key->ptr)) {
c->flags |= CLIENT_DIRTY_CAS;
} else if (replaced_with && dictFind(replaced_with->dict, key->ptr)) { c->flags |= CLIENT_DIRTY_CAS; }}}dictReleaseIterator(di);
}
Copy the codeHere we need to explain the mechanism of Watch.
0 x04 Watch mechanism
4.1 watch command
The Redis Watch command is used to monitor a key (or keys) and if the key (or keys) is changed by another command before the transaction is executed, the transaction will be interrupted
Syntax The syntax of the redis Watch command is as follows: Watch key [key…
Validation:
Start two Redis clients, client 1 and client 2.
-
- In client 1, set a value first
redis 127.0. 01.:6379> set number 10
OK
12
Copy the code-
- On client 1, enable Watch.
redis 127.0. 01.:6379> watch number
OK
12
Copy the code-
- Client 1 starts a transaction and changes this value
redis 127.0. 01.:6379> multi
OK
redis 127.0. 01.:6379> set number 100
QUEUED
redis 127.0. 01.:6379> get number
QUEUED
redis 127.0. 01.:6379>
1234567
Copy the codeNote that do not exec execution at this time
-
- Client 2, modify this value
redis 127.0. 01.:6379> set number 500
OK
12
Copy the code-
- For client 1, run
execperform
- For client 1, run
redis 127.0. 01.:6379> exec
(nil)
redis 127.0. 01.:6379> get number
"500"
1234
Copy the codeFound to be nil, execution failed, client 1 gets the value modified by client 2.
The logic is as follows:
Redis Client 1 Redis Server Redis Client 2
+ + +
| | |
| | |
| | |
v | |
set number 10 +-------------> | |
+ v |
| number = 10 |
| + |
| | |
v start watch | |
watch number +--------------> | |
+ | |
| | |
| | |
v begin traction | |
multi ---------------> | |
+ | |
| | |
| | |
v | |
set number 100 | |
+ | |
| | |
| | |
v | v
get number +<---------------+ set number 500
+ v +
| number = 500 |
| + |
v exec will fail | |
exec +-----------------> | |
+ | |
| nil | |
| | |
v | |
v |
get number <---------+ number = 500 |
+ + |
| | |
+ v +
Copy the code4.2 Mechanism Description
2 Redis transactions
Redis guarantees that all commands in a transaction are either executed or none are executed. If the client is disconnected before the EXEC command is sent, Redis empties the transaction queue and all commands in the transaction are not executed. Once the client sends the EXEC command, all the commands will be executed, even if the client is disconnected after that, because all the commands to be executed are recorded in Redis.
In addition, Redis transactions ensure that commands within a transaction are executed sequentially without being inserted by other commands. Imagine client A needs to execute several commands, and client B sends A command. If no transaction is used, client B’s command may be inserted into client A’s commands. If you don’t want this to happen, you can also use transactions.
4.2.2 Rollback Is not required
Redis’ Watch + Multi is actually an optimistic lock.
If there are multiple commands in a transaction and one command is wrong, none of the commands in the transaction will be executed. Therefore, unlike mysql transactions, redis transactions do not roll back when they are executed. Even if an error occurs, the result of a previously executed command will not be rolled back because there is no need to roll back.
With the optimistic lock function provided by WATCH, at the moment of EXEC, if the WATCH key has been changed, all commands between MULTI and EXEC will not be executed, and rollback is not required.
4.2.3 Prompt Failure
When client A and client B execute A piece of code at the same time, since the transaction execution is serial, assuming that client A is executed before CLIENT B, client A will be deleted from the list with the watch key and all clients in the list will be set to CLIENT_DIRTY_CAS when the execution is completed. Later, when B executes, the transaction finds that B is in CLIENT_DIRTY_CAS state and terminates the transaction with a failure return.
4.3 Watch source
This will add watch
Add a watch key to a client using watchCommand, and finally insert the watchedkey in watched_keys.
/* watch command */
void watchCommand(client *c) {
int j;
if (c->flags & CLIENT_MULTI) {
addReplyError(c,"WATCH inside MULTI is not allowed");
return;
}
for (j = 1; j < c->argc; j++)
watchForKey(c,c->argv[j]);
addReply(c,shared.ok);
}
typedef struct watchedKey {
robj *key;
redisDb *db;
} watchedKey;
/* watch a key */
void watchForKey(client *c, robj *key) {
list *clients = NULL;
listIter li;
listNode *ln;
watchedKey *wk;
/* Check whether the key is already watch. If so, return */
// Create an iterator
listRewind(c->watched_keys,&li);
// Walk through the client already watch key
while((ln = listNext(&li))) {
wk = listNodeValue(ln);
// If the key already exists, return directly
if (wk->db == c->db && equalStringObjects(key,wk->key))
return; /* Key already watched */
}
/ * is not watch, to continue the deal with * /
// Get the client list of the current key in the hash table
clients = dictFetchValue(c->db->watched_keys,key);
// If not, create a linked list for storage
if(! clients) { clients =listCreate(a);dictAdd(c->db->watched_keys,key,clients);
incrRefCount(key);
}
// Add the current client to the end of the list
listAddNodeTail(clients,c);
/* Maintain watch_keys list in client */
wk = zmalloc(sizeof(*wk));
wk->key = key;
wk->db = c->db;
incrRefCount(key);
listAddNodeTail(c->watched_keys,wk);
}
Copy the codeThe client uses watched_keys to monitor a set of keys:
+----------------------+
| client |
| | +------------+ +-------------+
| | | wk | | wk |
| watched_keys +--------> | key 1|... | key n | | | | db1 | | db n |
+----------------------+ +------------+ +-------------+
Copy the code4.3.2 Running Commands
Specifically:
- If the client status is set to CLIENT_DIRTY_CAS before the command is executed, the transaction is terminated without executing the command in the transaction queue.
- If a problem is found during the multi command execution, exit the traversal, call discardTransaction, and set the client flags plus CLIENT_DIRTY_CAS.
Details are as follows:
/* exec command */
void execCommand(client *c) {
int j;
robj **orig_argv;
int orig_argc;
struct redisCommand *orig_cmd;
int must_propagate = 0; /* Need to propagate MULTI/EXEC to AOF / slaves? * /
int was_master = server.masterhost == NULL;
// If multi is not executed, return
if(! (c->flags & CLIENT_MULTI)) {addReplyError(c,"EXEC without MULTI");
return;
}
/* * Key * Processing client state two states will terminate the transaction directly, CLIENT_DIRTY_CAS => When the watch key is touched * 2. CLIENT_DIRTY_EXEC => When the client enlists a command that does not exist */
if (c->flags & (CLIENT_DIRTY_CAS|CLIENT_DIRTY_EXEC)) {
addReply(c, c->flags & CLIENT_DIRTY_EXEC ? shared.execaborterr :
shared.nullmultibulk);
discardTransaction(c);
goto handle_monitor;
}
/* Execute the command */ in the queue
// Empty the watch key stored in the current client and the client node in the hash table
unwatchAllKeys(c); /* Unwatch ASAP otherwise we'll waste CPU cycles */
orig_argv = c->argv;
orig_argc = c->argc;
orig_cmd = c->cmd;
addReplyMultiBulkLen(c,c->mstate.count);
// Execute commands in the queue
for (j = 0; j < c->mstate.count; j++) {
c->argc = c->mstate.commands[j].argc;
c->argv = c->mstate.commands[j].argv;
c->cmd = c->mstate.commands[j].cmd;
/* ACL permissions are also checked at the time of execution in case * they were changed after the commands were ququed. * /
int acl_errpos;
int acl_retval = ACLCheckCommandPerm(c,&acl_errpos);
if (acl_retval == ACL_OK && c->cmd->proc == publishCommand)
acl_retval = ACLCheckPubsubPerm(c,1.1.0,&acl_errpos);
if(acl_retval ! = ACL_OK) {char *reason;
switch (acl_retval) {
case ACL_DENIED_CMD:
reason = "no permission to execute the command or subcommand";
break;
case ACL_DENIED_KEY:
reason = "no permission to touch the specified keys";
break;
case ACL_DENIED_CHANNEL:
reason = "no permission to publish to the specified channel";
break;
default:
reason = "no permission";
break; }}else {
// Call the relevant command
// If it involves modifying the command, the state of the client with the watch key in the hash table will be set to CLIENT_DIRTY_CAS regardless of whether the value is changed
call(c,server.loading ? CMD_CALL_NONE : CMD_CALL_FULL);
serverAssert((c->flags & CLIENT_BLOCKED) == 0);
}
/* Commands may alter argc/argv, restore mstate. */
c->mstate.commands[j].argc = c->argc;
c->mstate.commands[j].argv = c->argv;
c->mstate.commands[j].cmd = c->cmd;
}
c->argv = orig_argv;
c->argc = orig_argc;
c->cmd = orig_cmd;
discardTransaction(c);
handle_monitor:
/* Send EXEC to clients waiting data from MONITOR. We do it here * since the natural order of commands execution is actually: * MUTLI, EXEC, ... commands inside transaction ... * Instead EXEC is flagged as CMD_SKIP_MONITOR in the command * table, and we do it here with correct ordering. */
if (listLength(server.monitors) && ! server.loading)replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
}
/* Clear the current transaction data */
void discardTransaction(client *c) {
freeClientMultiState(c);
initClientMultiState(c);
c->flags &= ~(CLIENT_MULTI|CLIENT_DIRTY_CAS|CLIENT_DIRTY_EXEC);
unwatchAllKeys(c);
}
Copy the codeThe logic is shown below:
- The Client monitors a list of keys;
- When Redis DB fails to run multi, flags is set to CLIENT_DIRTY_CAS.
- When the client obtains the key and finds that the flag is set, it will not execute the commands in the transaction queue.
+-------------------+
| client |
| | +-------------+ +--------------+
| | 1 | wk | | wk |
| watched_keys +--------> | key 1|... | key n | | | | db1 | | db n |
| ^ | +-------------+ +--------------+
| | |
| | 3 | +----------------------+
| | | | Redis DB |
| | | | |
| + | 2 set CLIENT_DIRTY_CAS when error | |
| flags <--------------------------------------------+ execCommand(multi)|
| | | |
+-------------------+ +----------------------+
Copy the code0 x05 summary
Redis SWAPDB is a good command, but in the case of a business, it is important to deal with transactions.
0xEE Personal information
★★★★ Thoughts on life and technology ★★★★★
Wechat official account: Rosie’s Thoughts
If you want to get a timely news feed of personal articles, or want to see the technical information of personal recommendations, please pay attention.
0 XFF reference
SWAPDB index index
Redis SWAPDB command
What is the Watch mechanism of Redis?
Analysis of Redis Watch mechanism
— Distributed lock based on Redis Watch mechanism