Highly concurrent atomicity operations (Redis+Lua)

“This is the fourth day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021.”

What is Lua?

Lua is a compact, lightweight, extensible scripting language with the following features:

  • Lightweight: The source package has only the core library and is small after compilation.
  • Efficient: written by C, fast to start and run.
  • Embedded: Can be embedded in a variety of programming languages or systems to run, improve the flexibility of static languages.

Why does Redis use LUA?

  1. atomic: Multiple operations of Redis are combined into a script, and then executed as a whole. During the execution of the script, there will be no resource competition.
  2. Reduced Network traffic: Combine multiple commands into a lua script and redis executes the script uniformly.
  3. reusability: The script sent by the client is permanently stored in Redis, which means that other clients can reuse the script to complete the same logic.

Lua syntax introduction

EVAL script numkeys key [key ...]  arg [arg ...]Copy the code
  • Script: The argument is a Lua script. A script need not (and should not) be defined as a Lua function.
  • Numkeys: Specifies the number of key parameters.
  • key [key …] : represents the redis key, starting with the third argument to EVAL, and represents the redis key used in the script.
    • In Lua, these key-name parameters can be accessed with a base address of 1 through the global KEYS array (KEYS[1], KEYS[2], and so on).
  • arg [arg …] : represents an input parameter to Lua, which is accessed in Lua through the ARGV array of global variables similar to KEYS (ARGV[1], ARGV[2], and so on).

    • Note that lua’s array coordinates don’t start at 0, they start at 1!

Redis manages Lua scripts

The command role
EVAL script numkeys key [key …] arg [arg …] Execute the Lua script
EVALSHA sha1 numkeys key [key …] arg [arg …] Execute the Lua script
Redis -cli -a password –eval Lua script path key [key…] , arg [arg]… ARGV[1] can't get KEYS[2]For example, redis-cli -a 123456 –eval./ redis_CompareandSet. Lua userName, Zhangsan lisi
SCRIPT exists sha1 [sha1… This command is used to check whether SHA1 has been loaded into Redis memory
SCRIPT FLUSH Remove all scripts from the script cache
SCRIPT KILL Kill the currently running Lua script
SCRIPT LOAD script Adds to the script cache, but does not execute the script immediately 
127.0. 01.:6379> script load "return {KEYS[1],KEYS[2],ARGV[1],ARGV[2]}"
"a42059b356c875f0717db19a51f6aaca9ae659ea"
127.0. 01.:6379> script exists a42059b356c875f0717db19a51f6aaca9ae659ea
1) (integer) 1
127.0. 01.:6379>
Copy the code
127.0. 01.:6379> EVALSHA  a42059b356c875f0717db19a51f6aaca9ae659ea 2  key1 key2 ljw1 ljw2
1) "key1"
2) "key2"
3) "ljw1"
4) "ljw2"
127.0. 01.:6379> EVAL  "return {KEYS[1],KEYS[2],ARGV[1],ARGV[2]}" 2  key1 key2 ljw1 ljw2
1) "key1"
2) "key2"
3) "ljw1"
4) "ljw2"
Copy the code
127.0. 01.:6379>  script kill
(error) NOTBUSY No scripts in execution right now.
127.0.0.1:6379> script flush
OK
127.0.0.1:6379> script exists a42059b356c875f0717db19a51f6aaca9ae659ea
1) (integer) 0
Copy the code

A more detailed description of the above script:

  1. Eval is the keyword of Redis
  2. The contents of the double quotes represent lua scripts
  3. 2 represents the number of numkeys parameters, that is, the number of keys
  4. Key1 and key2 represent KEYS[1], inputs to KEYS[2]
  5. Ljw1 ljw2 is an input parameter to ARGV[1],ARGV[2]

5. Case analysis

Modify the user name. If the user does not exist in Redis, add the user name. If the user exists, modify the user name

    /** * Change the user name *@paramUid User ID *@paramUname User name */
    @GetMapping(value = "/updateUser")
    public void updateUser(Integer uid,String uname) {
        String key="user:"+uid;
        // Optimization point: the first redis request is sent
        String old=this.stringRedisTemplate.opsForValue().get(key);
        if(StringUtils.isEmpty(old)){
            // Optimization point: send redis request for the second time
            this.stringRedisTemplate.opsForValue().set(key,uname);
            return;
        }
        if(old.equals(uname)){
            log.info("{} need not be modified", key);
        }else{
            log.info("{} changed from {} to {}", key,old,uname);
            // Optimization point: send redis request for the second time
            this.stringRedisTemplate.opsForValue().set(key,uname); }}Copy the code

Problem analysis

The above code, while seemingly simple, still has a bit of a performance bottleneck in the case of high concurrency, mainly sending 2 Redis requests in terms of performance.

So how do you optimize?

We can use Lua technology to combine 2 Redis requests into one.

Solution optimization: Lua script

-- Successful setting returns1Return without setting0-- If Redis doesn't find it, just write it inif redis.call('get', KEYS[1]) == nil then
   redis.call('set', KEYS[1], ARGV[1]);
   return 1End - If the old value does not equal the new value, set the new valueif redis.call('get', KEYS[1]) ~= ARGV[1]  then
   redis.call('set', KEYS[1], ARGV[1]);
   return 1
else
   return 0
end
Copy the code

Linux command line to execute lua scripts

Save the above code as compareandSet.lua

ARGV./redis-cli --eval compareAndSet. Lua user: eval compareAndSet.101 , ljw101
Copy the code
  1. Eval tells redis-CLI to execute the following lua script, compareandSet. lua directory location
  2. User :101 is the key that Redis operates on, which can be obtained by using KEYS[1] in the Lua script
  3. “,” ljw101 after the comma is the lua argument, which can be retrieved from the Lua script with ARGV[1]
  4. Unlike in redis-CLI, there is no need to specify the number of KEYS, but KEYS and ARGV arguments need to be separated by a comma, with at least one space before and after the comma. Otherwise, an error is reported

ARGV, KEYS, ARGV, ARGV, ARGV, ARGV, ARGV, ARGV, ARGV

The execution effect is as follows:

[root@node2 src]# ./redis-cli  --eval compareAndSet.lua user:101 , ljw101
(integer) 1  // Set the value to redis when it is not found
[root@node2 src]# ./redis-cli  --eval compareAndSet.lua user:101 , ljw101
(integer) 0  // The second time, the old value is the same as the new value, returns 0
Copy the code

SpringBoot integrates lua scripts

Step 1: Write a Lua file and store it in Resources/Lua

To store the following content in resources / / compareAndSet lua lua

-- Successful setting returns1Return without setting0-- If Redis doesn't find it, just write it inif redis.call('get', KEYS[1]) == nil then
   redis.call('set', KEYS[1], ARGV[1]);
   return 1End - If the old value does not equal the new value, set the new valueif redis.call('get', KEYS[1]) ~= ARGV[1]  then
   redis.call('set', KEYS[1], ARGV[1]);
   return 1
else
   return 0
end
Copy the code

Step 2: Create a Lua script object

Create DefaultRedisScript object, used to hold the lua script, the resources / / compareAndSet lua lua script content, storage in DefaultRedisScript object.

@Configuration
public class LuaConfiguration {
  @Bean
    public DefaultRedisScript<Long> compareAndSetScript(a) {
        DefaultRedisScript<Long> redisScript = new DefaultRedisScript<>();
        redisScript.setScriptSource(new ResourceScriptSource(new ClassPathResource("lua/compareAndSet.lua")));
        redisScript.setResultType(Long.class);
        returnredisScript; }}Copy the code

Step 3: SpringBoot executes the Lua script

@Resource
private DefaultRedisScript<Long> compareAndSetScript;

@GetMapping(value = "/updateuserlua")
public void updateUserLua(Integer uid,String uname) {
    String key="user:"+uid;
    // Set the key of redis
    List<String> keys = Arrays.asList(key);
    The execute method takes 3 arguments. The first argument is a Lua script object, the second argument is a key list, and the third argument is an array of Lua parameters
    Long n = this.stringRedisTemplate.execute(this.compareAndSetScript, keys, uname);
    if (n == 0) {
        log.info("{} need not be modified", key);
    } else {
        log.info("{} changed to {}", key,uname); }}Copy the code

Step 4: Experience

http://127.0.0.1:8080/doc.html

2021-11-03 15:50:25.145  INFO 11160 --- [nio-9090-exec-1] c.a.r.c.CompareAndSetController          : user:1002Modified to LJW2021-11-03 15:50:45.538  INFO 11160 --- [nio-9090-exec-3] c.a.r.c.CompareAndSetController          : user:1002Don't need to modify2021-11-03 15:50:49.248  INFO 11160 --- [nio-9090-exec-2] c.a.r.c.CompareAndSetController          : user:1002Don't need to modify2021-11-03 15:51:59.796  INFO 11160 --- [nio-9090-exec-5] c.a.r.c.CompareAndSetController          : user:1002Modified to LJW - helloCopy the code

Redis distributed cache family

  • Redis Distributed Cache (1) – Redis Installation (Linux and Docker)
  • Redis Distributed cache (ii) – RDB and AOF
  • SpringBoot integrates Mybatis-Plus,Redis and Swagger
  • Redis distributed cache (4) – SpringCache integrates redis
  • Redis Distributed Cache (5) — Common command (String)
  • Redis Distributed Cache (6) — Article read volume PV Solution (String)
  • Redis Distributed Cache (7) — Distributed Global ID Solution (String)
  • The article continues to be updated…