An overview of the

Redis provides three ways to package multiple client commands and send them to the server for execution: Pipelining, Transactions, and Lua Scripts. This article will not discuss the basic knowledge of the three methods in detail, but will discuss the advantages, limitations and atomicity of the three methods

Pipelining

The Redis pipeline is the simplest among the three. When the client needs to execute multiple Redis commands, the client can send the commands to the server through the pipeline to reduce the impact of Round Trip Time (RTT) on performance. For example, we use the NC command to send two instructions to the Redis server

$ printf "INCR x\r\nINCR x\r\n" | nc localhost 6379
:1
:2
Copy the code

As you can see, the pipeline simply concatenates multiple commands, separating them with a newline character (/r/n), and not adding a start/end flag bit before the first command or after the last

After receiving multiple commands from the pipeline, the Redis server will keep executing the commands and cache the execution results of the commands until the last command is executed, and then all the execution results of the commands are returned to the client at a time

The advantage of Pipelining

Pipelining has two advantages in terms of performance:

  • Multiple commands are packaged and sent to the server at a time, reducing the number of network calls between the client and the serverRTT 
  • Context switch is avoided. When the client/server needs to read or write data from the network, a system call will be generated. System call is a very time-consuming operation, in which the program is designed to switch from user state to kernel state, and then from kernel state to user state. When we execute 10redisCommand, there will be 10 user to kernel context switches, but if we usePipeiningWhen multiple commands are packaged and sent to the server at once, only one context switch occurs

Pipelining atomic

As we all know, redis executes commands in a single thread, so all commands in Redis are atomic, which means that Redis does not stop in the middle of executing one command to execute another

But Pipelining is not atomic. Imagine that two clients, client1 and Client2, simultaneously send Pipelining commands to the Redis server. Each Pipelining command contains five REDis commands. Redis can guarantee that commands in the Client1 pipe are always executed sequentially, as are commands in the Client2 pipe, which are always executed in the order passed into the pipe

butredisThere are no guaranteesclient1All commands in the pipe are executed before being executedclient2Commands in a pipe, and therefore commands on the server might be executed in the following orderThis behavior shows thatPipeliningExecution does not block the server. Even if theclient1Sent multiple instructions to the clientPipeliningAnd other clients will not be blocked because the instructions they send can be inserted intoPipeliningThe middle to perform

Pipelining limitations

Only after all the Pipelining commands have been executed does the server send the results back to the client as an array. Pipelining can be executed if any of the commands in Pipelining fail

Take the following example

$ printf "SET name huangxy\r\nINCR name\r\nGET name\r\n" | nc localhost 6379
+OK
-ERR value is not an integer or out of range
$6
huangxy
Copy the code

Pipelining does not stop at the second command in Pipelining, which returns an error message when all operations are completed

This feature of Pipelining causes extra care when directives in Pipelining need to read the key set by previous directives because the value of the key can be modified by other clients. The results of Pipelining are often not what we expect

Pipelining use scenarios

  • There are performance requirements
  • Multiple instructions need to be sent to the server
  • The result of the previous command is not required as input to the next command

Transactions

Transactions in Redis are a little different from the concept of transactions that we learned about when we studied relational databases. The transaction mechanism in Redis is mainly used to queue multiple commands and finally decide whether to execute all commands in the transaction

Unlike pipes, transactions use special commands to mark the start and end of a transaction (MULTI, EXEC, DISCARD). The server can also queue commands in a transaction (so that the client can send one command at a time). In addition, some third-party libraries like to cache the command of a transaction in the client and then optimize it by sending the entire transaction in a pipe

Advantages of transactions

Transactions provide the WATCH command, which allows us to implement CAS capabilities, such as the INCR command through transactions

WATCH mykey
val = GET mykey
val = val + 1
MULTI
SET mykey $val
EXEC
Copy the code

Atomicity of transactions

Redis transactions have atomicity. When a transaction is executing, the server blocks other commands it receives and only executes the following commands when the transaction is complete. Therefore, transactions have atomicity

Transaction limitations

Like Pipelining, the results of multiple commands in a transaction are returned to the client only when the transaction is completed, so the client cannot obtain the results of its commands while the transaction is still running

If one of the commands in the transaction fails, there are two possibilities:

  • When a syntax error occurs, executeEXECThe transaction will be discarded and will not be executed
  • When a runtime error occurs (operating on the wrong data type),redisError messages are cached, subsequent commands are executed, and the execution results of all commands are returned to the client (error messages are also returned). This means thatredisThere is no rollback mechanism in the transaction

Transaction usage scenario

  • Multiple commands need to be executed atomically
  • The execution results of mid-transaction commands are not required to orchestrate subsequent commands

The Lua script

Redis introduced support for Lua scripting starting with version 2.6. By embedding the Lua environment in the server, Redis clients can directly use Lua scripts to execute multiple Redis commands on the server side atomically

Advantages of Lua scripts

Unlike Pipelining and transactions, inside the script, we can retrieve the results of intermediate commands and process them accordingly (such as if judgments).

local key = KEYS[1]
local new = ARGV[1]

local current = redis.call('GET', key)
if (current == false) or (tonumber(new) < tonumber(current)) then
  redis.call('SET', key, new)
  return 1
else
  return 0
end
Copy the code

In addition, the Redis server also supports the cache of Lua scripts (scripts executed using SCRIPT LOAD or EVAL are cached by the server). You can use the EVALSHA command to call the cached scripts next time, saving bandwidth

Atomicity of Lua scripts

Lua scripts are as atomic as transactions, and when the script is executed, the command received by the server is blocked

Limitations of Lua scripts

Lua scripts do not have too many functions. However, it is important to note that Lua scripts block the execution of other commands during execution. Therefore, do not write time-consuming processing logic in scripts

Usage scenarios of Lua scripts

  • Multiple commands need to be executed atomically
  • Intermediate values are needed to combine subsequent commands
  • Intermediate values are needed to orchestrate subsequent commands
  • Often used to extendredisFunction to implement commands that meet your own business scenarios

Reference documentation

  • Redis. IO/switchable viewer/pipe…
  • Redis. IO/switchable viewer/tran…
  • Redis. IO/commands/ev…
  • Rafaeleyng. Making. IO/redis – pipel…
  • Redis Design and Implementation by Huang Jianhong

Scan code to focus on me, learn together, progress together