background

The development of activity registration business involves the problem of activity number limit. When the amount of concurrency comes up, many people will submit the registration information at the same time, which will lead to the inaccurate number of activity registration and affect the business, as shown in the following figure:

The reason for the problem is that the accuracy of the current number is not ensured during the setting operation, that is, the consistency between the current number of registered people queried and the database is not ensured, resulting in two concurrent operations on the client being overwritten

Traditional database VS NoSql

mysql

For the above scenario, if the registration number field is stored in the mysql database, a common optimistic locking mechanism (Compare and Set CAS) can be used to reduce read/write lock conflicts and ensure data consistency. The implementation scheme is as follows

Will the original operation SQL code

update act set num=#{numNew} where actId=#{actId}

Copy the code

Instead of

update act set num=#{numNew} where actId=#{actId} and num=#{numOld}

Copy the code

That is, the assignment operation can be performed only when the queried data is consistent with the current database data. Otherwise, the assignment operation fails

redis

If you use Redis, the number of participants will be stored in the memory in the form of key-value pairs, and the business code will operate on the number of participants in the memory. Compared with mysql, Redis is more efficient and will not cause a great delay (if mysql is used to record the number of participants when there is a large amount of concurrency, CAS solution will cause many client operation failure, user experience is not good), but using Redis, it does not have good transaction support, the above mysql solution can not be well used in Redis, so how to design redis lock, share resources (number of registered activities) operation, is the problem to be solved

Description of the command used

Before we design the Redis lock, we need to introduce a few commands that will be used

SETNX

SET key to value. If key does not exist, in this case it is equivalent to the SET command, which returns 1. When key is present, do nothing and return the value 0.

watch && MULTI

Watch: indicates that all specified keys are monitored and conditionally executed in a transaction (optimistic locking)

MULTI: Marks the start of a transaction block. Subsequent instructions will be executed as an atom during EXEC execution

When the two are used together, the key is first monitored by watch. If any monitored key is modified by other clients when the EXEC command is invoked to execute the transaction, the whole transaction will not be executed and failure will be returned directly. The following table:

time The client A Client B
T1 WATCH name
T2 MULTI
T3 SET name owen
T4 SET name tom
T5 EXEC

At T4, client B modifies the value of the name key. When client A performs EXEC at T5, Redis will find that the monitored key name has been modified, so client A’s transaction will not be executed, but will return A failure directly.

GETSET

GETSET key Value returns the old value value and then sets the new value of the key

Redis basically solves the ideas and problems encountered

Here are the basic ideas for using redis locks

Note: The example uses the spring-data-redis library. The setnx command changes to setIfAbsent and returns true or false

private StringRedisTemplate stringRedisTemplate;



public Boolean setConcurrentLock(String key) throws InterruptedException {

    ValueOperations<String, String> ops = stringRedisTemplate.opsForValue();

while (! ops.setIfAbsent(key, "lock"))) {

        TimeUnit.MILLISECONDS.sleep(3);

    }

    return true;

}



public void deleteConcurrentLock(String key) {

    stringRedisTemplate.delete(key);

}

Copy the code

As shown above, the setnx command is used to obtain the redis lock. If the lock is occupied, false is returned and the loop continues until the lock is deleted and the value can be assigned successfully. Then the lock can be obtained and the shared resource can be locked.

However, it is obvious that while has the possibility of an infinite loop deadlock when:

Thread 1 obtains the lock, thread 2 and thread 3 execute a while loop waiting for the lock to be deleted. If thread 1 suddenly hangs and fails to delete the lock, thread 2 and thread 3 will execute an infinite loop and deadlock

The solution is to set timeout on the lock to prevent unlimited loops.



private StringRedisTemplate stringRedisTemplate;



public Boolean setConcurrentLock(String key, long expireTime) throws InterruptedException {

    ValueOperations<String, String> ops = stringRedisTemplate.opsForValue();

//expireTime specifies the lock timeout period

while (! ops.setIfAbsent(key, String.valueOf(System.currentTimeMillis() + expireTime))) {

        Long expire = Long.parseLong(ops.get(key));

// Determine whether timeout occurs

if (expire ! = null && expire < System.currentTimeMillis()) {

// getSet gets the old time and sets the new timeout

            Long oldExpire = Long.parseLong(ops.getAndSet(key, String.valueOf(System.currentTimeMillis() + expireTime)));

if (oldExpire ! = null && oldExpire < System.currentTimeMillis()) {

                break;

            }

        } 

        TimeUnit.MILLISECONDS.sleep(3);

    }

    return true;

}



public void deleteConcurrentLock(String key) {

  stringRedisTemplate.delete(key);

}

Copy the code

If the lock fails to be obtained, enter the while loop to determine whether the timeout time has expired. If the judgment is true, it proves that the lock has timed out. Therefore, run the getset command to obtain the old time and set a new timeout period. If the old time expires, the lock is successfully obtained and the loop is broken

However, adding timeout control here still has problems, as shown in the following scenario

Scene 1:

Thread 1 acquires the lock and hangs. Thread 2 and thread 3 enter the while loop and determine that the lock has timed out. Thread 2 first executes getSet and returns the timeout set by thread 1. Thread 3 executes getset and returns the timeout set by thread 2. It does not time out, but thread 3 resets the timeout

Scene 2:

If thread 2 holds the lock for a timeout but the operation is not completed, the lock is reset by thread 3 and becomes the lock of thread 3. If thread 2 executes del directly after the lock is completed, the lock of thread 3 will be deleted

Redis final practice plan

The final practice version of the modified code for the two problems listed above is as follows:

private StringRedisTemplate stringRedisTemplate;

public static ThreadLocal<String> holder = new ThreadLocal<>();



public Boolean setConcurrentLock(String key, long expireTime) throws InterruptedException {

    ValueOperations<String, String> ops = stringRedisTemplate.opsForValue();

while (! ops.setIfAbsent(key, String.valueOf(System.currentTimeMillis() + expireTime))) {

        stringRedisTemplate.watch(key);

        Long expire = Long.parseLong(ops.get(key));

if (expire ! = null && expire < System.currentTimeMillis()) {

            stringRedisTemplate.multi();

            Long oldExpire = Long.parseLong(ops.getAndSet(key, String.valueOf(System.currentTimeMillis() + expireTime)));

if (stringRedisTemplate.exec() ! = null && oldExpire ! = null && oldExpire < System.currentTimeMillis()) {

                break;

            }

        } else {

            stringRedisTemplate.unwatch();

        }

        TimeUnit.MILLISECONDS.sleep(3);

    }

    holder.set(ops.get(key));

    return true;

}



public void deleteConcurrentLock(String key) {

    ValueOperations<String, String> ops = stringRedisTemplate.opsForValue();

    Long expire = Long.valueOf(ops.get(key));

    if(exprie.equals(holder.get())){

        stringRedisTemplate.delete(key);

    }

    holder.remove();

}

Copy the code

In scenario 1 above, thread 2 and thread 3 will watch the lock and execute the getSet operation in the transaction. If thread 2 finishes the transaction and changes the lock time, thread 3 will fail to execute the transaction command because the lock is modified. Setting thread 2 timeout is not overridden, solving scenario 1 problem

For scenario 2, in order to prevent the thread that has timed out from deleting the lock of another thread that is executing, the variable ThreadLock is introduced, and the timeout time set by this thread is put into ThreadLock. If the time fetched from Redis changes during deletion, it proves that the thread has timed out and the time has been reset by another thread. There is no need to delete the lock. The last thing to note is that you need to manually delete a ThreadLocal when you determine that the lock is enough to delete. This prevents the thread pool in the Web server from reusing threads and causing the reuse of ThreadLocal.

conclusion

This practice is based on a single redis server lock (if the project is deployed on multiple machines, it can be called redis distributed lock). However, in the Redis cluster architecture, if the master node goes down, there will be obvious race-condition because the redis master/slave replication is asynchronous. Redis documentation provides a solution: RedLock, later on when the opportunity to practice learning…