Wechat search “code road mark”, point attention not lost! If you think it helps, give it a thumbs up!
In concurrent scenarios, multiple processes or threads share read and write resources. Ensure that the access to resources is mutually exclusive. In the stand-alone system, we can use Java and send package API, synchronized keyword and other ways to solve; However, in distributed systems, these methods are no longer applicable, and we need to implement distributed locks ourselves.
Common distributed lock implementation schemes include: based on database, based on Redis, based on Zookeeper. As part of Redis, this article will talk about distributed lock implementation based on Redis.
Analysis and Implementation
Problem analysis
Distributed locks share a common purpose with locks built into the JVM: they allow applications to access or operate on shared resources in the expected order and prevent multiple threads from operating on the same resource at the same time, resulting in chaotic and uncontrollable system performance. Often used for commodity inventory deduction, coupon deduction and other scenarios.
In theory, distributed locks must meet at least the following requirements to ensure lock security and effectiveness:
- Mutual exclusion: Only one thread can acquire a lock at a time.
- Deadlock-free: After a thread obtains a lock, it must be able to release the lock. Even if the application breaks down after the thread obtains the lock, the lock can be released within a specified time.
- Locking and unlocking must be the same thread;
In terms of implementation, distributed lock is generally divided into three steps:
- A – Operation right to obtain resources;
- B – Perform operations on resources;
- C – Operation right to release resources;
Both Java built-in locks and distributed locks, as well as whichever distributed implementation is used, are based on two steps A and C. Redis is naturally friendly for implementing distributed locks for the following reasons:
- Command processing stage Redis uses single thread processing, the same key can be processed by only one thread at the same time, there is no multi-thread race problem.
SET key value NX PX millisecondsCommand to add a key with an expiration time when no key exists to support secure locking.- The Lua script and DEL command provide reliable support for secure unlock.
Code implementation
- Maven rely on
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
<version>${your-spring-boot-version}</version>
</dependency>
Copy the code- The configuration file
Add the following to application.properties, a standalone Redis instance.
spring.redis.database=0
spring.redis.host=localhost
spring.redis.port=6379
Copy the code- RedisConfig
@Configuration
public class RedisConfig {
// Define a RedisTemplate
@Bean
@SuppressWarnings("all")
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory)
throws UnknownHostException {
,>
RedisTemplate<String, Object> template = new RedisTemplate<String,
Object>();
template.setConnectionFactory(factory);
// Json serialization configuration
Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
ObjectMapper om = new ObjectMapper();
om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
jackson2JsonRedisSerializer.setObjectMapper(om);
// Serialization of String
StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
// Key uses String serialization
template.setKeySerializer(stringRedisSerializer);
// The hash key also uses String serialization
template.setHashKeySerializer(stringRedisSerializer);
// Value serialization uses Jackson
template.setValueSerializer(jackson2JsonRedisSerializer);
// The hash value serialization method uses Jackson
template.setHashValueSerializer(jackson2JsonRedisSerializer);
template.afterPropertiesSet();
returntemplate; }}Copy the code- RedisLock
@Service
public class RedisLock {
@Resource
private RedisTemplate<String, Object> redisTemplate;
/** * lock, maxWait milliseconds **@paramLockKey lockKey *@paramLockValue lockValue *@paramTimeout Lock duration (ms) *@paramMaxWait Lock wait time (ms) *@returnTrue - success, false- failure */
public boolean tryAcquire(String lockKey, String lockValue, int timeout, long maxWait) {
long start = System.currentTimeMillis();
while (true) {
// Try locking
Boolean ret = redisTemplate.opsForValue().setIfAbsent(lockKey, lockValue, timeout, TimeUnit.MILLISECONDS);
if(! ObjectUtils.isEmpty(ret) && ret) {return true;
}
// Calculate the waiting time
long now = System.currentTimeMillis();
if (now - start > maxWait) {
return false;
}
try {
Thread.sleep(200);
} catch (Exception ex) {
return false; }}}/** * release lock **@paramLockKey lockKey *@paramLockValue lockValue *@returnTrue - success, false- failure */
public boolean releaseLock(String lockKey, String lockValue) {
/ / the lua script
String script = "if redis.call('get',KEYS[1]) == ARGV[1] then return redis.call('del',KEYS[1]) else return 0 end";
DefaultRedisScript<Long> redisScript = new DefaultRedisScript<>(script, Long.class);
Long result = redisTemplate.opsForValue().getOperations().execute(redisScript, Collections.singletonList(lockKey), lockValue);
returnresult ! =null && result > 0L; }}Copy the code- The test case
@SpringBootTest
class RedisDistLockDemoApplicationTests {
@Resource
private RedisLock redisLock;
@Test
public void testLock(a) {
redisLock.tryAcquire("abcd"."abcd".5 * 60 * 1000.5 * 1000);
redisLock.releaseLock("abcd"."abcd"); }}Copy the codeSafe hidden trouble
Many students (including me) use the above implementation in their daily work, which seems safe:
- use
setThe commandNX,PXOption to lock, ensure lock mutually exclusive, avoid deadlock; - Use lua script to unlock, preventing other threads from unlocking;
- Lock and unlock commands are atomic operations.
Set appendfsync=always to Redis on standalone.
But there may be problems in sentinel mode and cluster mode, why?
Sentinel mode and cluster mode are based on master-slave architecture. Data is synchronized between master and slave through command propagation, while command propagation is asynchronous.
Therefore, it is possible for the master node to break down without notifying the slave node of the success of data writing on the master node.
When the slave node is promoted to the new master node through failover, other threads have the opportunity to re-lock successfully, resulting in the mutually exclusive condition of distributed locks not being met.
The official RedLock
In cluster mode, security is guaranteed if all nodes in a cluster run stably and no failover occurs. However, no system can guarantee 100% stability, and distributed locks based on Redis must be fault-tolerant.
Because master/slave synchronization is based on asynchronous replication, sentinel mode and cluster mode cannot meet this requirement. To this end, Redis author specifically proposed a solution — RedLock (Redis Distribute Lock).
Design ideas
According to the official documentation, the design of RedLock is introduced.
First of all, N (N>=3) independently deployed Instances of Redis are required, without the need for master/slave replication, failover and other technologies.
To obtain the lock, the client will follow the following process:
- Get the current time (ms) as the start time start;
- Lock requests are made to all N nodes in sequence using the same key and random value. When a lock is set to each instance, the client uses an expiration time (less than the automatic release time of the lock). For example, if the lock is automatically released in 10 seconds, the timeout should be 5-50 milliseconds. This is to prevent the client from wasting too much time on an instance that has gone down: if the Redis instance goes down, the client processes the next instance as quickly as possible.
- Cost (cost=start-now) is calculated by the client. The current client is considered to be successfully locked only when the client successfully locks more than half of the instances and the entire time is less than the TTL.
- If the lock is successfully added to the client, the validTime of the lock is as follows: validTime=ttl-cost.
- If the client fails to lock (maybe less than half of the instances have successfully acquired the lock, or maybe the time spent exceeded TTL), then the client should try to unlock all instances (even if the client just thought the lock failed).
The design idea of RedLock continues the voting scheme of various scenarios inside Redis. Multiple instances are locked separately to solve the race problem. Although locking consumes time, it eliminates the security problem under the master-slave mechanism.
Code implementation
The official recommended Java implementation is Redisson, which has reentrant features and is implemented in accordance with RedLock, supporting independent instance mode, cluster mode, master-slave mode, sentinel mode, etc. The API is relatively simple and easy to use. The following is an example (directly from the test case) :
@Test
public void testRedLock(a) throws InterruptedException {
Config config = new Config();
config.useSingleServer().setAddress("Redis: / / 127.0.0.1:6379");
final RedissonClient client = Redisson.create(config);
// Get the lock instance
final RLock lock = client.getLock("test-lock");
/ / lock
lock.lock(60 * 1000, TimeUnit.MILLISECONDS);
try {
// Pretend to do something
Thread.sleep(50 * 1000);
} catch (Exception ex) {
ex.printStackTrace();
} finally {
/ / unlocklock.unlock(); }}Copy the codeThe Redisson package isso good that we can use it as if we were using Java’s built-in lock, and the code could not be less concise. For Redisson source analysis, there are many articles on the web you can find.
The full text summary
Distributed lock is a common way to solve concurrency problems in our development process, Redis is just one way to achieve.
The key is to understand the principles behind locking and unlocking, as well as the core issues that need to be addressed to implement distributed locking, and consider what features can be supported by the middleware we are using. With that in mind, implementation isn’t a problem.
With RedLock in mind, can we implement distributed locking in our own applications? Welcome to communicate!