Redis transaction principles

The transaction summary

Database transactions (transactions for short) usually consist of a sequence of read/write operations to the database. Contains the following two purposes:

1. Provides a way for the database operation sequence to recover from a failure to a normal state, and also provides a way for the database to maintain consistency even in an abnormal state.

2. When multiple applications concurrently access a database, an isolation method can be provided between these applications to prevent their operations from interfering with each other.

When a transaction is submitted to a database management system (DBMS), the DBMS needs to ensure that all operations in the transaction are successfully completed and the results are permanently stored in the database. If some operations in the transaction are not successfully completed, all operations in the transaction need to be rolled back to the state before the transaction execution. At the same time, this transaction has no impact on the execution of the database or other transactions, and all transactions seem to run independently.

“– Wikipedia

Redis transactions

Redis implements transactions through MULTI, EXEC, DISCARD, and WATCH commands. They allow one set of commands to be executed at a time, and to do this, Redis makes two guarantees:

All commands in the transaction are serialized and executed sequentially. During a Redis transaction, a request from another client is never executed. This ensures that the command is executed as a single isolated operation.

2. Either all commands are executed or none are executed, so Redis transactions are atomic. The EXEC command triggers the execution of all commands in a transaction, so if a client loses connection to the server in a transaction before the EXEC command is invoked, nothing is done. When using AOF, if the Redis server crashes or is forcibly killed by the system administrator, only part of the operation may be registered. Redis will detect this on reboot and will exit with an error. The redis-check-aof tool can be used to repair the AOF file, removing some transactions from the file so that the server can be restarted.

Transaction using

The MULTI command starts the transaction, and subsequent commands are cached in an array until Redis receives the EXEC command to execute the transaction.

> MULTI
OK
> INCR foo
QUEUED
> INCR bar
QUEUED
> EXEC
1) (integer) 1
2) (integer) 1
Copy the code

When the client sends MULTI to Redis, Redis turns on the CLIENT_MULTI bit of the flags for the client state information.

void multiCommand(client *c) {... c->flags |= CLIENT_MULTI; . }Copy the code

In the Redis command handler:

int processCommand(client *c) {...if(c->flags & CLIENT_MULTI && c->cmd->proc ! = execCommand && c->cmd->proc ! = discardCommand && c->cmd->proc ! = multiCommand && c->cmd->proc ! = watchCommand) { queueMultiCommand(c); addReply(c,shared.queued); }else{... }... }Copy the code

As long as it’s notMULTI,EXEC,DISCARD.WATCHEach command is appended to the client’s cache array of status information when receivedEXECAfter the command, Redis starts executing the cached commands sequentially.

EXECAfter the transaction execution order:

When the transaction completes, Redis clears the cache array, completes the transaction, and returns the result of each command.

EXEC error before

If an error occurs before the EXEC command is executed, such as a command error, parameter error, client disconnection, etc., Redis will call the flagTransaction function to enable the CLIENT_DIRTY_EXEC bit of the flags for the client state information:

/* Flag the transaction as DIRTY_EXEC so that EXEC will fail. * Should be called every time there is an error while queueing a command. */
void flagTransaction(client *c) {
    if (c->flags & CLIENT_MULTI)
        c->flags |= CLIENT_DIRTY_EXEC;
}
Copy the code

When received, the EXEC command checks whether the flags CLIENT_DIRTY_EXEC bit is on. If so, Redis will call the discardTransaction function to empty the transaction cache array and return an error message.

>MULTI
"OK"
>INCR foo
"QUEUED"
>INCR bar
"QUEUED"
>INCRB bar2
"ERR unknown command 'incrb'"
>INCR bar2
"QUEUED"
>EXEC
"EXECABORT Transaction discarded because of previous errors."
Copy the code

An error occurred during the EXEC command execution. Procedure

Redis skips the wrong command and continues to execute the next command in the transaction.

>MULTI
+OK
>SET a abc
+QUEUED
>LPOP a
+QUEUED
>EXEC
*2
+OK
-ERR Operation against a key holding the wrong kind of value
>GET a
"abc"
Copy the code

Unlike MySQL, Redis transactions do not roll back. About this, the official website has explained:

Redis commands can fail only if called with a wrong syntax (and the problem is not detectable during the command queueing), or against keys holding the wrong data type: this means that in practical terms a failing command is the result of a programming errors, and a kind of error that is very likely to be detected during development, and not in production.

Redis is internally simplified and faster because it does not need the ability to roll back.

An argument against Redis point of view is that bugs happen, however it should be noted that in general the roll back does not save you from programming errors. For instance if a query increments a key by 2 instead of 1, or increments the wrong key, there is no way for a rollback mechanism to help. Given that no one can save the programmer from his or her errors, and that the kind of errors required for a Redis command to fail are unlikely to enter in production, we selected the simpler and faster approach of not supporting roll backs on errors.

This command

When we want to DISCARD a transaction, we can do so through the DISCARD command.

>MULTI
"OK"
>INCR foo
"QUEUED"
>INCR bar
"QUEUED"
>DISCARD
"OK"
>GET foo
null
Copy the code

The DISCARD command disables the CLIENT_MULTI, CLIENT_DIRTY_CAS, and CLIENT_DIRTY_EXEC bits of client status flags (CLIENT_DIRTY_CAS is described in the WATCH command) and clears the transaction cache array to reclaim resources.

Here the transaction is only serialized execution of the command, and there is no isolation.

client-1>MULTI
"OK"
client-1>INCR foo
"QUEUED"
client-1>INCR bar client-2>INCR foo
"QUEUED"                 "1"
client-1>EXEC
 1)  "OK"
 2)  "2"
 3)  "OK"
 4)  "1"
 5)  "OK"
 client-1>GET foo
 "2"
Copy the code

This is clearly wrong.

A transaction does not differ much from pipelining in that it executes multiple commands at once, except that pipelining is cached at the client and sent to Redis at once.

As we all know, the isolation of transactions is generally achieved using locking mechanism, Redis uses WATCH to implement optimistic locking.

WATCH command

WATCH key [key …]

Redis will save the keys after WATCH in watched_keys dictionary of server db. The key of the dictionary is each key, and the value of the dictionary is a list. Each element in the list is all the clients of WATCH key.

And the client status message has awatched_keysThe list of, save each clientWATCHKey to prevent duplicationWATCH, determine whether to focus on a key,UNWATCHAnd so on.

WATCH mykey
val = GET mykey
val = val + 1
MULTI
SET mykey $val
EXEC
Copy the code

Redis executes the signalModifiedKey hook function every time a modification command is executed,

/* Every time a key in the database is modified the function * signalModifiedKey() is called. */
void signalModifiedKey(redisDb *db, robj *key) {
    touchWatchedKey(db,key);
}
Copy the code

The watched_keys list of client state information is traversed to determine whether the client is aware of this key. If so, enable the CLIENT_DIRTY_CAS bit of client state information flags.

When the EXEC command is executed, the flags CLIENT_DIRTY_CAS bit is determined to be enabled. If enabled, the transaction is abandoned and an error is reported.

Related command source

/ / MULTI command
void multiCommand(client *c) {
    if (c->flags & CLIENT_MULTI) {
        addReplyError(c,"MULTI calls can not be nested");
        return;
    }
    // Enable CLIENT_MULTI bit
    c->flags |= CLIENT_MULTI;
    addReply(c,shared.ok);
}

/ / this command
void discardCommand(client *c) {
    if(! (c->flags & CLIENT_MULTI)) { addReplyError(c,"DISCARD without MULTI");
        return;
    }
    // Transaction resource reclamation
    discardTransaction(c);
    addReply(c,shared.ok);
}

void discardTransaction(client *c) {
    // Reclaim the transaction cache array
    freeClientMultiState(c);
    // reinitialize the cache array
    initClientMultiState(c);
    // Close the CLIENT_MULTI, CLIENT_DIRTY_CAS, and CLIENT_DIRTY_EXEC bits
    c->flags &= ~(CLIENT_MULTI|CLIENT_DIRTY_CAS|CLIENT_DIRTY_EXEC);
    // unwatch all keys
    unwatchAllKeys(c);
}

/ / the EXEC command
void execCommand(client *c) {.../* Check if we need to abort the EXEC because: * 1) Some WATCHed key was touched. * 2) There was a previous error while queueing commands. * A failed EXEC in the first  case returns a multi bulk nil object * (technically it is not an error but a special behavior), while * in the second an EXECABORT error is returned. */
    // Check whether CLIENT_DIRTY_CAS and CLIENT_DIRTY_EXEC are enabled
    if (c->flags & (CLIENT_DIRTY_CAS|CLIENT_DIRTY_EXEC)) {
        ...
        // Transaction resource reclamationdiscardTransaction(c); . }... }/ / WATCH command
void watchCommand(client *c) {
    int j;

    if (c->flags & CLIENT_MULTI) {
        addReplyError(c,"WATCH inside MULTI is not allowed");
        return;
    }
    for (j = 1; j < c->argc; j++)
        watchForKey(c,c->argv[j]);
    addReply(c,shared.ok);
}

/* Watch for the specified key */
void watchForKey(client *c, robj *key) {.../* Check if we are already watching for this key */
    listRewind(c->watched_keys,&li);
    while((ln = listNext(&li))) {
        wk = listNodeValue(ln);
        if (wk->db == c->db && equalStringObjects(key,wk->key))
            return; /* Key already watched */
    }
    /* This key is not already watched in this DB. Let's add it */
    clients = dictFetchValue(c->db->watched_keys,key);
    if(! clients) { clients = listCreate(); dictAdd(c->db->watched_keys,key,clients); incrRefCount(key); } listAddNodeTail(clients,c);/* Add the new key to the list of keys watched by this client */
    wk = zmalloc(sizeof(*wk));
    wk->key = key;
    wk->db = c->db;
    incrRefCount(key);
    listAddNodeTail(c->watched_keys,wk);
}

/ / UNWATCH command
void unwatchCommand(client *c) {
    // Retrieve the watch keys
    unwatchAllKeys(c);
    // Close the CLIENT_DIRTY_CAS bit
    c->flags &= (~CLIENT_DIRTY_CAS);
    addReply(c,shared.ok);
}
Copy the code

Afterword.

The implementation principles of Redis transactions can be understood in conjunction with Pipelining and PUB/SUB. Without WATCH, a set of commands is executed at once in much the same way as pipelining, which caches commands at the client and sends them to the server for execution at once.

With the WATCH command, the maintenance logic for WATCHED Keys is similar to the maintenance logic for pub/sub. Take a look at the Redis Pub/Sub implementation I wrote earlier. Give yourself a wave of advertising 😁

The transaction summary

Redis transactions

Transaction using

EXEC error before

An error occurred during the EXEC command execution. Procedure

This command

WATCH command

Related command source

Afterword.

Related Posts

GoFrame frame (RK-boot) : enable TLS/SSL

Concurrent programming Voliatile

Must see! MySQL > Select * from ‘MySQL’;