Tencent interviewer: “database transaction mechanism understand?”

“Inner monologue: Small, not ACID, in a moment of thought, I interview but a technical expert, not so simple question.”

Cheng Xuyuan: balabala… spoke with great confidence and composure.

Tencent interviewer: “Redis transaction understanding? Its transaction mechanism can achieve ACID property?”

Cheng Xuyuan: “scratch your head, this… I know lua script can implement transactions…”

Tencent interviewer: “good, go back and so on notice.”

Code brother, I followed you to learn Redis series and got a lot of offers, but I failed in the end with “how does Redis implement transactions?” On this question.

Let’s break it down step by step:

  1. What is transaction ACID?
  2. How does Redis implement transactions?
  3. What properties can Redis transactions implement?
  4. Lua script implementation.

What is transactional ACID

Ghost blow the lamp “Yunnan Bug Valley” touch gold school commander has a saying called “together is living, divided is dead”, in order to find the mu dust bead they three clear division of labor, work together to be successful.

A Transaction is a unit of concurrency control, a sequence of operations that either all or none of them execute.

“It is an inseparable unit of work.”

When a transaction executes, it provides special attribute guarantees:

  • Atomicity: a transaction in which multiple operations must complete, or none of them. Welcome to comment in the comments section);

  • Consistency: After a transaction is executed, the integrity constraint of the database is not broken and the sequence of transaction execution is legal data state.

    Database integrity constraints include but are not limited to:

    • Entity integrity (e.g. row primary keys exist and are unique);
    • Column integrity (e.g., field type, size, length to meet requirements)
    • Foreign key constraint;
    • User-defined integrity (e.g., the sum of two account balances should remain the same before and after a transfer).

  • Isolation: Operations within a transaction are isolated from other transactions, and concurrently executed transactions cannot interfere with each other.

    What matters is the interaction between different transactions, and strict isolation corresponds to Serializable in the isolation level.

  • Durability: Once a transaction is committed, all modifications are permanently saved in the database, data is not lost even if the system crashes and restarts.

How does Redis implement the transaction mechanism after understanding ACID’s specific requirements?

How does Redis implement transactions

MULTI, EXEC, DISCARD, and WATCH commands are the basis for Redis to implement transactions.

Redis transaction execution consists of three steps:

  1. Start a transaction;
  2. Command to join the ranks;
  3. Execute transaction or discard;

Explicitly start a transaction

The client explicitly starts a transaction with the MULTI command, and subsequent commands are queued in the cache and not actually executed.

The command team

The client sends the sequence of instructions to be executed in the transaction to the server.

Note that although the instructions are sent to the server, the Redis instance simply stores the sequence of instructions in a command queue and does not execute them immediately.

Execute transaction or discard

The client sends a command to the server to commit or discard the transaction, and then Redis executes the specific instruction sent in step 2 or the command to clear the queue and abandon execution.

Redis only needs to call EXEC to schedule queue command execution.

You can also DISCARD the commands stored in the queue in step 2.

Redis transaction case

Execute our sample code: try.redis. IO from an online debugging site

Normal execution

Execute a transaction process through MULTI and EXEC:

SET QUEUED > SET "order" "30" QUEUED > SET "number of articles" 666 QUEUED > GET QUEUED # EXEC 1) OK 2) OK 3) OK 4) "666"Copy the code

We see QUEUED as the result of each read/write operation, indicating that the thank you operation has been temporarily stored in the command queue and has not yet been executed.

When the EXEC command is executed, you can see the response data for each instruction.

Give up the transaction

DISCARD queue commands through MULTI and DISCARD:

SET "order:mobile" 100 OK "DECR "order:mobile" QUEUED # DISCARD OK # > GET "order:mobile" "100"Copy the code

Code, can Redis transactions guarantee ACID properties?

That’s a good question. Let’s analyze it.

Redis transactions satisfy ACID?

Redis transactions can execute multiple commands at once with three important guarantees:

  1. Batch instructions are queued temporarily before executing EXEC commands;
  2. After receiving the EXEC command, the transaction is executed. Any command in the transaction fails to be executed, and other commands are still executed.
  3. During transaction execution, commands submitted by other clients are not inserted into the current command execution sequence.

atomic

Is atomic performance guaranteed if an error occurs during transaction execution, Code-brother?

During a transaction, you may encounter two types of command errors:

  • In the implementationEXECThe command sent before the command itself is wrong. As follows:
    • Parameter number error;
    • Command name error, using a nonexistent command;
    • Out of memory (Redis instance usemaxmemoryInstructions to configure memory limits.
  • In the implementationEXECThe command may fail. For example, the data types of the command and operation do not match (a List operation is performed on a String value);
  • In the execution of a transactionEXECCommand. The Redis instance failed and the transaction failed.

EXEC reported an error before execution

When the queue is ordered, Redis reports an error and logs the error.

At this point, we can continue to submit commands.

After the EXEC command is executed, Redis will reject all submitted command operations and return the result that the transaction failed.

In this way, all commands in the transaction will no longer be executed, ensuring atomicity.

The following is an example of a transaction failure caused by an instruction enqueue error:

Send the first operation in the transaction, but Redis does not support this command. Error message 127.0.0.1:6379> PUT order 6 (error) ERR unknown command 'PUT', with args beginning with: 'order', '6', # send the second operation in the transaction, this operation is correct, Redis QUEUED the command > DECR b:stock QUEUED # Redis therefore rejects the > EXEC (error) EXECABORT Transaction discarded because of previous errors.Copy the code

EXEC reported an error after execution

When the transaction operation was enqueued, the data types of the command and operation did not match, but the Redis instance did not check for an error.

However, after executing the EXEC command, Redis actually executes the instructions, and an error is reported.

Redis will report an error, but the transaction will still execute the correct command, so the atomicity of the transaction is not guaranteed!

Why doesn’t Redis support rollback?

In fact, there is no rollback mechanism in Redis. Although Redis provides the DISCARD command.

However, this command can only be used to actively abandon transaction execution, temporarily empty the command queue, does not have the effect of rollback.

A fault occurred during EXEC execution

If Redis has AOF logging enabled, only part of the transactions will be logged to the AOF log.

We need to check the AOF log file using the redis-check-aof tool, which removes unfinished transactions from the AOF file.

In this way, after we use AOF to restore the instance, the transaction will not be executed again, thus ensuring atomicity.

Summary:

  • An error will be reported when the command enters the team, and transaction execution will be abandoned to ensure atomicity;
  • No error was reported when the command entered the team, but the actual execution reported error, does not guarantee atomicity;
  • If AOF logging is enabled, atomicity can be guaranteed.

consistency

Consistency is affected by the timing of error command and instance failure. According to the timing of two dimensions of command error instance failure, three cases can be analyzed.

Queue entry error reported before EXEC execution

Transactions are abandoned, so consistency is guaranteed.

An error is reported after EXEC execution

Incorrect execution will not be executed, correct instructions can be executed, consistency can be guaranteed.

Instance failure during EXEC execution

The instance will be restarted after failure, which is related to the way of data recovery. We will discuss the case according to whether the instance has RDB or AOF enabled.

If we do not enable RDB or AOF, then after the instance fails and restarts, the data is lost and the database is consistent.

If we use the RDB snapshot, because the RDB snapshot is not executed at transaction execution time.

Therefore, the results of transaction command operations are not saved to the RDB snapshot, and the data in the database is consistent when the RDB snapshot is used for recovery.

If we use THE AOF log and the instance fails before the transaction is logged to the AOF log, the database data recovered using the AOF log is consistent.

If only part of the operation is logged to the AOF, we can use redis-check-aof to clear the completed operation in the transaction, and the database will be consistent after recovery.

Isolation,

Transaction execution can be divided into two stages: command enqueue (before EXEC command execution) and command actual execution (after EXEC command execution).

Therefore, during the concurrent execution, we analyze the two phases in two cases:

  1. Concurrent operation inEXECBefore the command is executed, the isolation must passWATCHMechanism guarantee;
  2. Concurrent operation inEXECAfter command, isolation is guaranteed.

Jack, what is the WATCH mechanic?

Let’s focus on the first case: the command operations of a transaction are transient in the command queue while the EXEC command of a transaction has not yet been executed.

At this point, if there are other concurrent operations and the same key is changed, we need to see whether the transaction uses the WATCH mechanism.

The function of the WATCH mechanism is to monitor the value changes of one or more keys before the transaction execution. When the transaction invokes the EXEC command to execute, the WATCH mechanism will first check whether the monitored keys are modified by other clients.

If modified, transaction execution is abandoned to avoid breaking the isolation of the transaction.

At the same time, the client can execute the transaction again. At this point, if there are no concurrent modification of transaction data, the transaction can execute normally and isolation is guaranteed.

Don’t WATCH

Without the WATCH mechanism, concurrent operations prior to EXEC command execution read and write data.

When EXEC is executed, the data to be manipulated within a transaction has changed, and Redis does not isolate transactions.

Concurrent operations receive execution after EXEC

In the second case, because Redis uses a single thread to execute commands, and after the EXEC command is executed, Redis guarantees that all commands in the command queue are executed before any subsequent commands are executed.

So, in this case, concurrent operations do not break the isolation of the transaction.

persistence

If Redis does not use RDB or AOF, the persistence properties of the transaction are definitely not guaranteed.

If Redis uses RDB mode, then the data modified by the transaction is not guaranteed to persist if the instance goes down and the data is lost after a transaction is executed but before the next RDB snapshot is taken.

If Redis uses AOF mode, there will be data loss due to the three AOF mode configuration options no, Everysec and always.

Therefore, the persistence properties of transactions are not guaranteed.

No matter what persistence model Redis uses, the persistence properties of a transaction are not guaranteed.

conclusion

  • Redis has some atomicity, but does not support rollback.
  • Redis does not have the concept of consistency in ACID. (or that Redis didn’t design it)
  • Redis has isolation.
  • Redis does not guarantee persistence.

Redis’s transaction mechanism guarantees consistency and isolation, but not persistence.

However, because Redis itself is an in-memory database, persistence is not a required attribute, and we are more concerned with atomicity, consistency, and isolation.

Atomicity is a complicated case. If the command syntax used in a transaction is incorrect, atomicity is not guaranteed. In other cases, transactions can be atomically executed.

Good article recommendation

Why so fast

Redis Persistence: How do AOF and RDB ensure high availability of data

Redis High Availability: Principles of data consistency synchronization between master and slave architectures

Redis High Availability: Sentry Cluster principles

Redis High Availability: Cluster Cluster principle

Redis combat: skillfully use Bitmap to achieve hundreds of millions of massive data statistics

Redis actual combat: through the Geo type to achieve nearby people meet goddess

Redis new features: Multi-threaded model interpretation

What’s new in Redis 6.0: The revolution brought about by client caching