Mp.weixin.qq.com/s/UEeRx6epB…

A concept,

Idempotent is an interface that initiates the same request multiple times and must ensure that the operation can only be performed once, for example:

  • Order interface, cannot create order multiple times


  • Payment interface, repeated payment for the same order can only deduct money once


  • The Alipay callback interface, which may have multiple callbacks, must handle repeated callbacks


  • Common form submission interface, because of network timeout and other reasons, click submit many times, can only succeed once, etc


Second, common solutions

  1. Unique index – prevents new dirty data


  2. Token mechanism – Prevents page duplication


  3. Pessimistic locking — Locking (table or row) while fetching data


  4. Optimistic locking – Implemented based on the version number, verifies data at the moment it is updated


  5. Distributed locks – Redis (Jedis, Redisson) or ZooKeeper implementations


  6. State machine – State changes are determined when data is updated


Iii. Implementation of this paper

In this paper, the second way is adopted, that is, through redis + Token mechanism to achieve interface idempotency check


Four, the realization of ideas

To create a uniquely identified token for each request that needs to be idempotent, the token is first obtained and stored in Redis


When requesting an interface, put the token into the header or request the interface as a request parameter. The back-end interface checks whether the token exists in redis:


  • If the token exists, the service logic is processed and the token is deleted from redis. Then, if the request is repeated, the token has been deleted, and the verification fails, a do not repeat operation message is displayed


  • If no, the parameter is invalid or the request is repeated

V. Project Introduction

  • springboot

  • redis

  • The @APIidempotent annotation + interceptor intercepts the request

  • @controllerAdvice Handle global exceptions

  • Pressure measuring tool: Jmeter


Description:

  • This article focuses on the idempotence core implementation. Details about how SpringBoot integrates Redis, ServerResponse, ResponseCode and other details are not discussed in this article.


  • If you are interested, you can check out my Github project:

    https://github.com/wangzaiplus/springboot/tree/wxw

Six, code implementation

  1. pom

        Copy the code


  1. JedisUtil

package com.wangzaiplus.test.util; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; import redis.clients.jedis.Jedis; import redis.clients.jedis.JedisPool; @Component@Slf4jpublic class JedisUtil { @Autowired private JedisPool jedisPool; private JedisgetJedis() {        returnjedisPool.getResource(); } /** * set value ** @param key * @param value * @return     */    public String set(String key, String value) {        Jedis jedis = null;        try {            jedis = getJedis();            return jedis.set(key, value);        } catch (Exception e) {            log.error("set key:{} value:{} error", key, value, e);            returnnull; } finally { close(jedis); }} /** * Set value ** @param key * @param value * @param expireTime Expiration time, unit: s * @return     */    public String set(String key, String value, int expireTime) {        Jedis jedis = null;        try {            jedis = getJedis();            return jedis.setex(key, expireTime, value);        } catch (Exception e) {            log.error("set key:{} value:{} expireTime:{} error", key, value, expireTime, e);            returnnull; } finally { close(jedis); }} /** * Value ** @param key * @return     */    public String get(String key) {        Jedis jedis = null;        try {            jedis = getJedis();            return jedis.get(key);        } catch (Exception e) {            log.error("get key:{} error", key, e);            returnnull; } finally { close(jedis); } /** * delete key ** @param key * @return     */    public Long del(String key) {        Jedis jedis = null;        try {            jedis = getJedis();            return jedis.del(key.getBytes());        } catch (Exception e) {            log.error("del key:{} error", key, e);            returnnull; } finally { close(jedis); }} /** * Check whether the key exists ** @param key * @return     */    public Boolean exists(String key) {        Jedis jedis = null;        try {            jedis = getJedis();            return jedis.exists(key.getBytes());        } catch (Exception e) {            log.error("exists key:{} error", key, e);            returnnull; } finally { close(jedis); }} /** * Key Expiration time ** @param key * @param expireTime Expiration time (unit: s * @)return     */    public Long expire(String key, int expireTime) {        Jedis jedis = null;        try {            jedis = getJedis();            return jedis.expire(key.getBytes(), expireTime);        } catch (Exception e) {            log.error("expire key:{} error", key, e);            returnnull; } finally { close(jedis); }} /** * get the remaining time ** @param key * @return     */    public Long ttl(String key) {        Jedis jedis = null;        try {            jedis = getJedis();            return jedis.ttl(key);        } catch (Exception e) {            log.error("ttl key:{} error", key, e);            return null;        } finally {            close(jedis);        }    }    private void close(Jedis jedis) {        if (null != jedis) {            jedis.close();        }    }}Copy the code


  1. Custom annotation @APIidempotent

package com.wangzaiplus.test.annotation; import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; */ @target ({elementtype.method})@Retention(retentionPolicy.runtime)public */ @target ({elementtype.method})@Retention(retentionPolicy.runtime)public @interface ApiIdempotent {}Copy the code


  1. ApiIdempotentInterceptor interceptor

package com.wangzaiplus.test.interceptor; import com.wangzaiplus.test.annotation.ApiIdempotent; import com.wangzaiplus.test.service.TokenService; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.method.HandlerMethod; import org.springframework.web.servlet.HandlerInterceptor; import org.springframework.web.servlet.ModelAndView; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.lang.reflect.Method; */public class ApiIdempotentInterceptor implements HandlerInterceptor {@autoWired Private TokenService tokenService; @Override public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {if(! (handler instanceof HandlerMethod)) {return true;        }        HandlerMethod handlerMethod = (HandlerMethod) handler;        Method method = handlerMethod.getMethod();        ApiIdempotent methodAnnotation = method.getAnnotation(ApiIdempotent.class);        if(methodAnnotation ! = null) { check(request); // Idempotent check, pass the check, fail to throw an exception, and return a friendly message through unified exception processing}return true;    }    private void check(HttpServletRequest request) {        tokenService.checkToken(request);    }    @Override    public void postHandle(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, Object o, ModelAndView modelAndView) throws Exception {    }    @Override    public void afterCompletion(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, Object o, Exception e) throws Exception {    }}Copy the code


  1. TokenServiceImpl

package com.wangzaiplus.test.service.impl; import com.wangzaiplus.test.common.Constant; import com.wangzaiplus.test.common.ResponseCode; import com.wangzaiplus.test.common.ServerResponse; import com.wangzaiplus.test.exception.ServiceException; import com.wangzaiplus.test.service.TokenService; import com.wangzaiplus.test.util.JedisUtil; import com.wangzaiplus.test.util.RandomUtil; import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.text.StrBuilder; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import javax.servlet.http.HttpServletRequest; @Servicepublic class TokenServiceImpl implements TokenService { private static final String TOKEN_NAME ="token";    @Autowired    private JedisUtil jedisUtil;    @Override    public ServerResponse createToken() {        String str = RandomUtil.UUID32();        StrBuilder token = new StrBuilder();        token.append(Constant.Redis.TOKEN_PREFIX).append(str);        jedisUtil.set(token.toString(), token.toString(), Constant.Redis.EXPIRE_TIME_MINUTE);        return ServerResponse.success(token.toString());    }    @Override    public void checkToken(HttpServletRequest request) {        String token = request.getHeader(TOKEN_NAME);        if(stringutils. isBlank(token)) {// Header does not contain token token = request.getParameter(TOKEN_NAME);if(stringutils.isblank (token)) {// parameter throw new ServiceException(responsecode.illegal_argument. GetMsg ());  }}if(! jedisUtil.exists(token)) { throw new ServiceException(ResponseCode.REPETITIVE_OPERATION.getMsg()); } Long del = jedisUtil.del(token);if(del <= 0) { throw new ServiceException(ResponseCode.REPETITIVE_OPERATION.getMsg()); }}}Copy the code


  1. TestApplication

package com.wangzaiplus.test; import com.wangzaiplus.test.interceptor.ApiIdempotentInterceptor; import org.mybatis.spring.annotation.MapperScan; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.context.annotation.Bean; import org.springframework.web.cors.CorsConfiguration; import org.springframework.web.cors.UrlBasedCorsConfigurationSource; import org.springframework.web.filter.CorsFilter; import org.springframework.web.servlet.config.annotation.InterceptorRegistry; import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter; @SpringBootApplication@MapperScan("com.wangzaiplus.test.mapper")public class TestApplication extends WebMvcConfigurerAdapter { public static void main(String[] args) { SpringApplication.run(TestApplication.class, args); } /** * cross-domain * @return     */    @Bean    public CorsFilter corsFilter() {        final UrlBasedCorsConfigurationSource urlBasedCorsConfigurationSource = new UrlBasedCorsConfigurationSource();        final CorsConfiguration corsConfiguration = new CorsConfiguration();        corsConfiguration.setAllowCredentials(true);        corsConfiguration.addAllowedOrigin("*");        corsConfiguration.addAllowedHeader("*");        corsConfiguration.addAllowedMethod("*");        urlBasedCorsConfigurationSource.registerCorsConfiguration("/ * *", corsConfiguration);        returnnew CorsFilter(urlBasedCorsConfigurationSource); } @override public void addInterceptors(InterceptorRegistry registry) {// Interface idempotent interceptor registry.addInterceptor(apiIdempotentInterceptor()); super.addInterceptors(registry); } @Bean public ApiIdempotentInterceptorapiIdempotentInterceptor() {        return new ApiIdempotentInterceptor();    }}Copy the code


OK, so far, the verification code is ready to test and verify


7. Test and verification



  1. The TokenController that obtains the token

package com.wangzaiplus.test.controller; import com.wangzaiplus.test.common.ServerResponse; import com.wangzaiplus.test.service.TokenService; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; @RestController@RequestMapping("/token")public class TokenController {    @Autowired    private TokenService tokenService;    @GetMapping    public ServerResponse token() {        return tokenService.createToken();    }}Copy the code


  1. TestController, note the @APIidempotent annotation, which can be declared on methods that require idempotent validation

package com.wangzaiplus.test.controller; import com.wangzaiplus.test.annotation.ApiIdempotent; import com.wangzaiplus.test.common.ServerResponse; import com.wangzaiplus.test.service.TestService; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; @RestController@RequestMapping("/test")@Slf4jpublic class TestController {    @Autowired    private TestService testService;    @ApiIdempotent    @PostMapping("testIdempotence")    public ServerResponse testIdempotence() {        return testService.testIdempotence();    }}Copy the code


  1. Access token



Check the redis




  1. Test interface security: Use JMeter test tool to simulate 50 concurrent requests, taking the token obtained in the previous step as parameter




  1. Header or parameter does not transmit the token, or the token value is empty, or the token value is incorrectly filled, so the verification fails. For example, the token value is “ABcd”.





Eight, pay attention (very important)




In the preceding figure, you cannot simply delete the token without checking whether the deletion succeeds, which may cause concurrency security problems


Because it is possible for multiple threads to reach line 46 at the same time before the token is removed, proceed further


If the delete result of jedisutil.del (token) is not verified, then the double commit problem will still occur, even though there is only one real delete operation, as reproduced below


Modify the code slightly:



The request again



If you look at the console




Although only one of the tokens was actually deleted, there was still a concurrency problem because the deletion result was not verified, so it must be verified


Nine,

In fact, the idea is very simple, is to ensure the uniqueness of each request, so as to ensure idempotent, through interceptor + annotation, do not need to write repeated code every request, in fact, can also use Spring AOP implementation, does not matter


If you have any questions or suggestions, please feel free to make them


Github

https://github.com/wangzaiplus/springboot/tree/wxw