1. What is idempotent

Idempotent is common in our programming

  • Select query natural idempotent
  • Delete Delete is idempotent. Delete the same number of times
  • Update Directly updates an idempotent value
  • Update Update the non-idempotent operation
  • Insert is a non-idempotent operation, one at a time

2. Causes

Due to repeated clicks or network retransmission:

  • Click the submit button twice;
  • Click the refresh button;
  • Using the browser back button to repeat the previous action, causing the form to be submitted repeatedly;
  • Submit the form repeatedly using browser history;
  • Browser duplicate HTTP please;
  • Nginx retransmission;
  • Try retransmission of distributed RPC;

Public number: Code ape technology column

3. Solutions

1. The front-end JS submission disable button can use some JS components

2. Use the Post/Redirect/Get mode

Page redirection is performed after submission, which is known as post-redirect-get (PRG) mode. In short, when the user submits the form, you perform a client-side redirect to the successful submission information page. This avoids the repeated submission caused by the user pressing F5 without the warning of the repeated submission of the browser form, and eliminates the same problem caused by pressing forward and back in the browser.

3. Store a special flag in the session

On the server side, generates a unique identifier, put it in the session, at the same time to write it into the form of hidden fields, and then to send the form page to the browser, the user input information, then click the submit on the server side, get the value of the hidden fields in the form, compared with a unique identifier of the session, equal specification is submitted for the first time, Process the request and remove the unique identifier from the session; Unequal means repeated submission and will not be processed.

4. Others use the header header to set the Cache control header cache-control, etc

It is not suitable for the application of mobile terminal APP

5. Use a database

Insert uses unique index UPDATE uses optimistic lock version version method

This efficiency under large data volumes and high concurrency depends on database hardware capabilities and can be targeted at non-core businesses

6. Use pessimistic locks

Use the select… For update, this is similar to synchronized locking insert or update, but it is inefficient to avoid deadlocks

It is recommended for small concurrency of single requests

7. Local locking (emphasis of this article)

Principle: Use the ConcurrentHashMap concurrent container putIfAbsent method, and ScheduledThreadPoolExecutor timing task, also can use guava cache mechanism, Gauva has content-MD5 generation of key with valid time of cache is also possible

Content-md5 refers to the MD5 value of the Body. MD5 is calculated only when the Body is not a Form Form. MD5 is directly encrypted for parameters and parameter names

MD5 is approximately unique in a range of classes that are considered unique and of course sufficient for low concurrency

The local lock applies only to applications in single-machine deployment

1. Configure annotations

import java.lang.annotation.*;

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface Resubmit {

    /** * Delay time how long after the delay can be submitted again **@return Time unit is one second
     */
    int delaySeconds(a) default 20;
}
Copy the code

2. Instantiate the lock

import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.codec.digest.DigestUtils;

import java.util.Objects;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

/ * * *@authorLijing * double commit lock */
@Slf4j
public final class ResubmitLock {


    private static final ConcurrentHashMap<String, Object> LOCK_CACHE = new ConcurrentHashMap<>(200);
    private static final ScheduledThreadPoolExecutor EXECUTOR = new ScheduledThreadPoolExecutor(5.new ThreadPoolExecutor.DiscardPolicy());


   // private static final Cache<String, Object> CACHES = CacheBuilder.newBuilder()
            // Maximum cache size is 100
   // .maximumSize(1000)
            // Set write cache expiration to 5 seconds
   // .expireAfterWrite(5, TimeUnit.SECONDS)
   // .build();


    private ResubmitLock(a) {}/** * Static inner class singleton mode **@return* /
    private static class SingletonInstance {
        private static final ResubmitLock INSTANCE = new ResubmitLock();
    }

    public static ResubmitLock getInstance(a) {
        return SingletonInstance.INSTANCE;
    }


    public static String handleKey(String param) {
        return DigestUtils.md5Hex(param == null ? "" : param);
    }

    /** * putIfAbsent is an atomic operation to ensure thread safety **@paramKey The corresponding key *@param value
     * @return* /
    public boolean lock(final String key, Object value) {
        return Objects.isNull(LOCK_CACHE.putIfAbsent(key, value));
    }

    /** * Delay release lock to control repeated commit ** within a short time@paramLock Whether to unlock *@paramKey The corresponding key *@paramDelaySeconds Delay time */
    public void unLock(final boolean lock, final String key, final int delaySeconds) {
        if(lock) { EXECUTOR.schedule(() -> { LOCK_CACHE.remove(key); }, delaySeconds, TimeUnit.SECONDS); }}}Copy the code

3. The AOP aspects

import com.alibaba.fastjson.JSONObject;
import com.cn.xxx.common.annotation.Resubmit;
import com.cn.xxx.common.annotation.impl.ResubmitLock;
import com.cn.xxx.common.dto.RequestDTO;
import com.cn.xxx.common.dto.ResponseDTO;
import com.cn.xxx.common.enums.ResponseCode;
import lombok.extern.log4j.Log4j;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.stereotype.Component;

import java.lang.reflect.Method;

/ * * *@ClassName RequestDataAspect
 * @DescriptionData is submitted repeatedly for verification *@Author lijing
 * @Date2019/05/16 17:05 * * /
@Log4j
@Aspect
@Component
public class ResubmitDataAspect {

    private final static String DATA = "data";
    private final static Object PRESENT = new Object();

    @Around("@annotation(com.cn.xxx.common.annotation.Resubmit)")
    public Object handleResubmit(ProceedingJoinPoint joinPoint) throws Throwable {
        Method method = ((MethodSignature) joinPoint.getSignature()).getMethod();
        // Get the annotation information
        Resubmit annotation = method.getAnnotation(Resubmit.class);
        int delaySeconds = annotation.delaySeconds();
        Object[] pointArgs = joinPoint.getArgs();
        String key = "";
        // Get the first argument
        Object firstParam = pointArgs[0];
        if (firstParam instanceof RequestDTO) {
            // Parse the parameters
            JSONObject requestDTO = JSONObject.parseObject(firstParam.toString());
            JSONObject data = JSONObject.parseObject(requestDTO.getString(DATA));
            if(data ! =null) {
                StringBuffer sb = new StringBuffer();
                data.forEach((k, v) -> {
                    sb.append(v);
                });
                // Generate encryption parameters using content_MD5 encryptionkey = ResubmitLock.handleKey(sb.toString()); }}/ / perform lock
        boolean lock = false;
        try {
            // Set the unlock key
            lock = ResubmitLock.getInstance().lock(key, PRESENT);
            if (lock) {
                / / release
                return joinPoint.proceed();
            } else {
                // Response repeated commit exception
                return newResponseDTO<>(ResponseCode.REPEAT_SUBMIT_OPERATION_EXCEPTION); }}finally {
            // Set the unlock key and unlock timeResubmitLock.getInstance().unLock(lock, key, delaySeconds); }}}Copy the code

4. Annotate use cases

@apiOperation (value = "save my thread interface ", notes =" Save my thread interface ")
    @PostMapping("/posts/save")
    @Resubmit(delaySeconds = 10)
    public ResponseDTO<BaseResponseDataDTO> saveBbsPosts(@RequestBody @Validated RequestDTO<BbsPostsRequestDTO> requestDto) {
        return bbsPostsBizService.saveBbsPosts(requestDto);
    }

Copy the code

So that’s the local lock for idempotent commits using Content-MD5 encryption so as long as the parameters don’t change, the encryption value of the parameters doesn’t change, and the key exists, the commit is blocked

Of course, some other signature verification can be used to generate a fixed signature during a certain submission and submit it to the back end. The unified signature can be used as the verification token of each submission and processed in the cache according to the back end resolution.

8. With distributed Redis locks (see others)

Add the dependency of starter- Web, starter- AOP, starter-data-redis to pom. XML

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-aop</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis</artifactId>
    </dependency>
</dependencies>
Copy the code

Property configuration Adds redis-related configuration items to the application.properites resource file

spring.redis.host=localhost
spring.redis.port=6379
spring.redis.password=123456
Copy the code

Main implementation methods:

Those familiar with Redis know that it is thread safe, we can easily implement a distributed lock using its features, For example, opsForValue().setifAbsent (key,value) returns true if there is no current key in the cache and vice versa;

After the cache, set an expiration time for the key to prevent the lock from being released due to system crash. So can we say that when we return true we think it acquired the lock, and when the lock is not released we throw an exception…

package com.battcn.interceptor;

import com.battcn.annotation.CacheLock;
import com.battcn.utils.RedisLockHelper;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.util.StringUtils;

import java.lang.reflect.Method;
import java.util.UUID;

/** * Redis solution **@author Levin
 * @since2018/6/12 0012 * /
@Aspect
@Configuration
public class LockMethodInterceptor {

    @Autowired
    public LockMethodInterceptor(RedisLockHelper redisLockHelper, CacheKeyGenerator cacheKeyGenerator) {
        this.redisLockHelper = redisLockHelper;
        this.cacheKeyGenerator = cacheKeyGenerator;
    }

    private final RedisLockHelper redisLockHelper;
    private final CacheKeyGenerator cacheKeyGenerator;


    @Around("execution(public * *(..) ) && @annotation(com.battcn.annotation.CacheLock)")
    public Object interceptor(ProceedingJoinPoint pjp) {
        MethodSignature signature = (MethodSignature) pjp.getSignature();
        Method method = signature.getMethod();
        CacheLock lock = method.getAnnotation(CacheLock.class);
        if (StringUtils.isEmpty(lock.prefix())) {
            throw new RuntimeException("lock key don't null...");
        }
        final String lockKey = cacheKeyGenerator.getLockKey(pjp);
        String value = UUID.randomUUID().toString();
        try {
            // If the lock is locked successfully, but the expiration time is set to invalid, then we will get false
            final boolean success = redisLockHelper.lock(lockKey, value, lock.expire(), lock.timeUnit());
            if(! success) {throw new RuntimeException("Duplicate submission");
            }
            try {
                return pjp.proceed();
            } catch (Throwable throwable) {
                throw new RuntimeException("System exception"); }}finally {
            // TODO needs to comment this code if it is demonstrated; Actually let goredisLockHelper.unlock(lockKey, value); }}}Copy the code

RedisLockHelper is called in API mode, which is more flexible

package com.battcn.utils;

import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisStringCommands;
import org.springframework.data.redis.core.RedisCallback;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.core.types.Expiration;
import org.springframework.util.StringUtils;

import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.regex.Pattern;

/** * needs to be defined as Bean **@author Levin
 * @since2018/6/15 0015 * /
@Configuration
@AutoConfigureAfter(RedisAutoConfiguration.class)
public class RedisLockHelper {


    private static final String DELIMITER = "|";

    /** ** Can be allocated by injection if the requirements are high
    private static final ScheduledExecutorService EXECUTOR_SERVICE = Executors.newScheduledThreadPool(10);

    private final StringRedisTemplate stringRedisTemplate;

    public RedisLockHelper(StringRedisTemplate stringRedisTemplate) {
        this.stringRedisTemplate = stringRedisTemplate;
    }

    /** * Get lock (there is deadlock risk) **@param lockKey lockKey
     * @param value   value
     * @paramTime Timeout period *@paramUnit Expiration unit *@return true or false
     */
    public boolean tryLock(final String lockKey, final String value, final long time, final TimeUnit unit) {
        return stringRedisTemplate.execute((RedisCallback<Boolean>) connection -> connection.set(lockKey.getBytes(), value.getBytes(), Expiration.from(time, unit), RedisStringCommands.SetOption.SET_IF_ABSENT));
    }

    /** * get lock **@param lockKey lockKey
     * @param uuid    UUID
     * @paramTimeout Indicates the timeout period *@paramUnit Expiration unit *@return true or false
     */
    public boolean lock(String lockKey, final String uuid, long timeout, final TimeUnit unit) {
        final long milliseconds = Expiration.from(timeout, unit).getExpirationTimeInMilliseconds();
        boolean success = stringRedisTemplate.opsForValue().setIfAbsent(lockKey, (System.currentTimeMillis() + milliseconds) + DELIMITER + uuid);
        if (success) {
            stringRedisTemplate.expire(lockKey, timeout, TimeUnit.SECONDS);
        } else {
            String oldVal = stringRedisTemplate.opsForValue().getAndSet(lockKey, (System.currentTimeMillis() + milliseconds) + DELIMITER + uuid);
            final String[] oldValues = oldVal.split(Pattern.quote(DELIMITER));
            if (Long.parseLong(oldValues[0]) + 1 <= System.currentTimeMillis()) {
                return true; }}return success;
    }


    / * * *@see <a href="http://redis.io/commands/set">Redis Documentation: SET</a>
     */
    public void unlock(String lockKey, String value) {
        unlock(lockKey, value, 0, TimeUnit.MILLISECONDS);
    }

    /** * delay unlock **@param lockKey   key
     * @paramUuid client(preferably unique key) *@paramDelayTime delayTime *@paramUnit Time unit */
    public void unlock(final String lockKey, final String uuid, long delayTime, TimeUnit unit) {
        if (StringUtils.isEmpty(lockKey)) {
            return;
        }
        if (delayTime <= 0) {
            doUnlock(lockKey, uuid);
        } else{ EXECUTOR_SERVICE.schedule(() -> doUnlock(lockKey, uuid), delayTime, unit); }}/ * * *@param lockKey key
     * @paramUuid client(preferably unique key) */
    private void doUnlock(final String lockKey, final String uuid) {
        String val = stringRedisTemplate.opsForValue().get(lockKey);
        final String[] values = val.split(Pattern.quote(DELIMITER));
        if (values.length <= 0) {
            return;
        }
        if (uuid.equals(values[1])) { stringRedisTemplate.delete(lockKey); }}}Copy the code