The amount that a single machine can carry is limited. There are tens of thousands of users. Basically, services are deployed in distributed clusters. Many times, you will encounter methods on the same resource. In this case, you need to lock. If it is a stand-alone version, you can use Java and other languages with concurrent synchronization processing. If multiple machines are deployed, an intermediary agent is required to do distributed locking.
Commonly used distributed locks are implemented in three ways.
- Redis based implementation (using redis atomic operation setnx to implement)
- Mysql based implementation (using mysql innoDB row lock to achieve, there are two ways, pessimistic lock and optimistic lock)
- Zookeeper-based implementation (implemented using ZK’s temporary sequential nodes)
At present, I have used Redis and mysql to achieve the lock, and it is applied in different online environments according to the application scenario. Zk implementation is more complex, and there is no application scenario, interested can refer to the “Zookeeper implementation distributed lock” in the stone of his mountain.
Talk about your experience.
There is no perfect technology, no master key and different application scenarios in different ways. The PRINCIPLE of CAP is consistency, availability and partition tolerance.
His shan zhishi
- Zookeeper implementation distributed lock: www.jianshu.com/p/5d12a0101…
- Several implementations of a distributed lock ~ : www.hollischuang.com/archives/17…
- Select for update cause deadlock analysis: www.cnblogs.com/micrari/p/8…
Distributed lock based on Redis cache
The implementation of the lock based on Redis is relatively simple, because redis is executed by a single thread, naturally with atomic operation, we can use the command setnx and expire to implement, Java version code reference is as follows:
package com.fenqile.creditcard.appgatewaysale.provider.util;
import com.fenqile.redis.JedisProxy;
import java.util.Date;
/** * User: Rudy Tan * Date: 2017/11/20 ** redis related */
public class RedisUtil {
/** * Get distributed lock **@paramKey string Cache key *@paramExpireTime Int Expiration time, in seconds *@returnBoolean true- lock seized, false- lock not seized */
public static boolean getDistributedLockSetTime(String key, Integer expireTime) {
try {
// Remove the failed lock
String temp = JedisProxy.getMasterInstance().get(key);
Long currentTime = (new Date()).getTime();
if (null! = temp && Long.valueOf(temp) < currentTime) { JedisProxy.getMasterInstance().del(key); }/ / lock contentions
Long nextTime = currentTime + Long.valueOf(expireTime) * 1000;
Long result = JedisProxy.getMasterInstance().setnx(key, String.valueOf(nextTime));
if (result == 1) {
JedisProxy.getMasterInstance().expire(key, expireTime);
return true; }}catch (Exception ignored) {
}
return false; }}Copy the codeChange the package name and get redis operation object to your own.
The basic steps are
- Check to see if this key is implemented every time you come in. Remove the failed lock if it fails
- Use the setnx atomic command to scramble for locks.
- Grab lock set expiration time.
Step 2 is the most important thing, why step 3? The lock may not be released because of some removal request from the thread that acquired the lock, so set a maximum lock time to avoid deadlocks. Why step 1? Redis may fail when setting expire. Failed to set expiration time, but the lock became permanent.
Online environment, step 1, step 3 problems have occurred. So you have to make a safety tackle.
Redis cluster deployment
In general, Redis uses master-slaves to solve single point problems. Multiple master-slaves form a large cluster and route different keys to different master-slaves through consistent hash algorithm.
Advantages and disadvantages of Redis lock:
Advantages: Redis is inherently in-memory and is usually deployed on multiple slices, so it has high concurrency control and can withstand a large number of requests. Disadvantages: Redis itself is a cache, there is a certain probability of inconsistent data requests.
Online, before, using Redis to do inventory counters, the prize distribution theory only issued 10, finally issued 14. Data consistency problems arise.
So after that, mysql database distributed locking was introduced.
Distributed lock based on mysql implementation.
Implement the first version
Before this, I searched a lot of articles on the Internet, which were basically by way of inserting, deleting or directly obtaining locks and counters through “select for update”. Specific can refer to the “distributed lock in several ways ~” about the database lock chapter.
In the beginning, I implemented the pseudo-code as follows:
public boolean getLock(String key){
select for update
if (Records exist){
update
}else {
insert
}
}
Copy the codeSelect for Update causes deadlock analysis. Select for Update causes deadlock analysis.
1. Physical deletion of online data is generally not allowed. 2. 3. If the AppClient hangs before releasing the lock, the lock persists and a deadlock occurs. 4. If you implement the redis counter (INCR DECR) in this way, there will be a lot of deadlock situations when records do not exist.
Therefore, consider introducing the concept of recording status fields and central locking.
Implement version 2
The database table design has been improved in the second version, as follows:
Lock table, single library single table
CREATE TABLE IF NOT EXISTS credit_card_user_tag_db.t_tag_lock (
Record the index --
Findex INT NOT NULL AUTO_INCREMENT COMMENT 'Increment index ID'.-- Lock information (key, counter, expiration time, record description)
Flock_name VARCHAR(128) DEFAULT ' ' NOT NULL COMMENT 'Lock name key value',
Fcount INT NOT NULL DEFAULT 0 COMMENT 'counter',
Fdeadline DATETIME NOT NULL DEFAULT '1970-01-01 00:00:00' COMMENT 'Lock Expiration time',
Fdesc VARCHAR(255) DEFAULT ' ' NOT NULL COMMENT 'Value/Description'.Record status and related events
Fcreate_time DATETIME NOT NULL DEFAULT '1970-01-01 00:00:00' COMMENT 'Creation time',
Fmodify_time DATETIME NOT NULL DEFAULT '1970-01-01 00:00:00' COMMENT 'Modification time',
Fstatus TINYINT NOT NULL DEFAULT 1 COMMENT 'Record status, 0: invalid, 1: valid'.-- Primary key (PS: total index cannot exceed 5)
PRIMARY KEY (Findex),
-- Unique constraint
UNIQUE KEY uniq_Flock_name(Flock_name),
-- Common index
KEY idx_Fmodify_time(Fmodify_time)
)ENGINE=INNODB DEFAULT CHARSET=UTF8 COMMENT 'credit card | and counter table | rudytan | 20180412';
Copy the codeIn this version, the concept of central locking is introduced in consideration of deadlock (gap lock contention) in the case of concurrent insertion of another lock.
The basic way is:
- Create a database based on SQL
- Create a record with Flock_name=”center_lock”.
- Select for update on “center_lock” before operating on other locks (for example, Flock_name=”sale_invite_lock”)
- “Sale_invite_lock” records its own additions, deletions, changes and checks.
Given that different companies have introduced different database operation packages, pseudo code is provided to make it easier to understand the pseudo code
// Start the transaction
@Transactional
public boolean getLock(String key){
// Get the central lock
select * from tbl where Flock_name="center_lock"
// Query key-related records
select for update
if (Records exist){
update
}else {
insert
}
}
Copy the code
/** * Initializes the record, if there is an update record, if there is no insert */
private LockRecord initLockRecord(String key){
// Check whether the record exists
LockRecord lockRecord = lockMapper.queryRecord(key);
if (null == lockRecord) {
// Record does not exist, create
lockRecord = new LockRecord();
lockRecord.setLockName(key);
lockRecord.setCount(0);
lockRecord.setDesc("");
lockRecord.setDeadline(new Date(0));
lockRecord.setStatus(1);
lockMapper.insertRecord(lockRecord);
}
return lockRecord;
}
/** * get lock, code snippet */
@Override
@Transactional
public GetLockResponse getLock(GetLockRequest request) {
// Check parameters
if(StringUtils.isEmpty(request.lockName)) {
ResultUtil.throwBusinessException(CreditCardErrorCode.PARAM_INVALID);
}
// Initialization of compatible parameters
request.expireTime = null==request.expireTime? 31536000: request.expireTime;
request.desc = Strings.isNullOrEmpty(request.desc)?"":request.desc;
Long nowTime = new Date().getTime();
GetLockResponse response = new GetLockResponse();
response.lock = 0;
// Get the central lock, initialize the record
lockMapper.queryRecordForUpdate("center_lock");
LockRecord lockRecord = initLockRecord(request.lockName);
// Failed to obtain the lock because the lock was not released or expired
if (lockRecord.getStatus() == 1
&& lockRecord.getDeadline().getTime() > nowTime){
return response;
}
/ / acquiring a lock
Date deadline = new Date(nowTime + request.expireTime*1000);
int num = lockMapper.updateRecord(request.lockName, deadline, 0, request.desc, 1);
response.lock = 1;
return response;
}
Copy the codeTo this point, the scheme can meet the needs of my distributed lock.
However, this scheme has a fatal problem, that is, all the records share a lock, and the concurrency is not high.
After the test, 50*100 threads were started for concurrent modification, and the average time of 5 times was 8 seconds.
Implement version 3
In scheme 2, there are requests sharing the same central lock with low concurrency. Based on the implementation principle of concurrentHashMap, the concept of segmental lock is introduced to reduce the lock granularity.
The basic way is:
- Create a database based on SQL
- Create 100 records Flock_name=”center_lock_xx” (xx is 00-99).
- When performing operations on other locks (for example, Flock_name=”sale_invite_lock”), find the corresponding center_lock_02 based on the CRC32 algorithm, and record the select for update for “center_lock_02” first
- “Sale_invite_lock” records its own additions, deletions, changes and checks.
The pseudocode is as follows:
// Start the transaction
@Transactional
public boolean getLock(String key){
// Get the central lock
select * from tbl where Flock_name="center_lock"
// Query key-related records
select for update
if (Records exist){
update
}else {
insert
}
}
Copy the code /** * get central lock Key */
private boolean getCenterLock(String key){
String prefix = "center_lock_";
Long hash = SecurityUtil.crc32(key);
if (null == hash){
return false;
}
// take the last two values in crc32
Integer len = hash.toString().length();
String slot = hash.toString().substring(len-2);
String centerLockKey = prefix + slot;
lockMapper.queryRecordForUpdate(centerLockKey);
return true;
}
/** * get lock */
@Override
@Transactional
public GetLockResponse getLock(GetLockRequest request) {
// Check parameters
if(StringUtils.isEmpty(request.lockName)) {
ResultUtil.throwBusinessException(CreditCardErrorCode.PARAM_INVALID);
}
// Initialization of compatible parameters
request.expireTime = null==request.expireTime? 31536000: request.expireTime;
request.desc = Strings.isNullOrEmpty(request.desc)?"":request.desc;
Long nowTime = new Date().getTime();
GetLockResponse response = new GetLockResponse();
response.lock = 0;
// Get the central lock, initialize the record
getCenterLock(request.lockName);
LockRecord lockRecord = initLockRecord(request.lockName);
// Failed to obtain the lock because the lock was not released or expired
if (lockRecord.getStatus() == 1
&& lockRecord.getDeadline().getTime() > nowTime){
return response;
}
/ / acquiring a lock
Date deadline = new Date(nowTime + request.expireTime*1000);
int num = lockMapper.updateRecord(request.lockName, deadline, 0, request.desc, 1);
response.lock = 1;
return response;
}
Copy the codeAfter the test, 50*100 threads were started for concurrent modification, and the average time of 5 seconds was 5 times. That’s almost double the improvement from version 2.
At this point, complete redis/mysql distributed lock, counter implementation and application.
The last
Select the following options based on application scenarios:
- High concurrency, no guarantee of data consistency: Redis lock/counter
- Low concurrency, data consistency: mysql lock/counter
- Low concurrency, no guarantee of data consistency: suit yourself
- High concurrency. Ensure data consistency: Redis lock/counter + mysql lock/counter.
Table data and records:
Welcome to my Jane book blog, grow together, progress together.
www.jianshu.com/u/5a327aab7…