Hello, I am Misty. SpringBoot old bird series articles have written four, each reading response is still good, that today continues to bring you the fifth series of old bird, to talk about how to limit the interface in SpringBoot project, what are the common limiting algorithms, how elegant limiting (AOP based).
First let’s look at why we need to limit traffic on interfaces.
Why do we need to limit the current?
The Internet system is usually faced with requests with large amounts of concurrent traffic. In case of an emergency (the most common scenario is second kill and purchase), the instantaneous traffic will directly break down the system and make it unable to provide services. One of the most common solutions to prevent this is to limit traffic, waiting, queuing, demoting, denial of service, etc., when a request reaches a certain number of concurrent requests or rates.
For example, 12306 ticketing system adopts traffic limiting in the face of high concurrency. Notifications often appear during peak traffic periods; There is a large queue, please try again later!”
What is current limiting? What are the traffic limiting algorithms?
Traffic limiting limits the number of requests in a certain period of time to ensure system availability and stability and prevent system slowdowns or downtime caused by traffic surges.
There are three common traffic limiting algorithms:
1. Current limiting of the counter
Counter flow limiting algorithm is the most simple and crude solution, mainly used to limit the total number of concurrent, such as database connection pool size, thread pool size, interface access concurrency, etc., are using counter algorithm.
For example, AomicInteger is used to count the number of concurrent executions. If the number exceeds the value, the request is rejected, indicating that the system is busy.
2. Leaky bucket algorithm
Bucket algorithm idea is very simple, we compared the water to be at the request of bucket to a system capacity limits, first into the water in the bucket, the water in the bucket in a certain flow rate, when the outflow rate is less than the rate of flow, due to the limited capacity of bucket, subsequent water directly into the overflow (rejected requests), to realize the current limit.
3. Token bucket algorithm
The principle of token bucket algorithm is also relatively simple, we can understand it as the registration of the hospital to see a doctor, only after getting the number can be diagnosed.
The system maintains a token bucket and puts tokens into the bucket at a constant speed. If a request comes in and wants to be processed, it needs to obtain a token from the bucket first. If no token is available in the bucket, the request will be denied service. Token bucket algorithm can limit requests by controlling bucket capacity and token issuing rate.
Implement traffic limiting based on Guava tool class
Google open source toolkit Guava provides RateLimiter, which realizes traffic limiting based on token bucket algorithm. It is very convenient to use and highly efficient. The implementation steps are as follows:
Step 1: Introduce the Guava dependency package
< the dependency > < groupId > com. Google. Guava < / groupId > < artifactId > guava < / artifactId > < version > 30.1 jre < / version > </dependency>Copy the codeStep 2: Add limiting logic to the interface
@slf4j @restController @requestMapping ("/limit") Public Class LimitController {/** * LimitController: */ Private final RateLimiter limiter = ratelimite.create (2.0); private DateTimeFormatter dtf = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"); @getMapping ("/test1") public String testLimiter() {@getMapping ("/test1") public String testLimiter() { Boolean tryAcquire = limiter. TryAcquire (500, timeunit.milliseconds); if (! TryAcquire) {log.warn(" Enter service degraded, time {}", localDatetime.now ().format(DTF)); Return "There is a large queue, please try again later!" ; } log.info(" token obtained successfully, time {}", localDatetime.now ().format(DTF)); Return "Request successful "; }}Copy the codeThe two core methods of RateLimiter, create() and tryAcquire(), are used above, as described below
- Acquire () acquires a token, which blocks until the token is acquired and returns the time it took to acquire the token
- Acquire (int permits) obtain a specified number of tokens, the method also blocks, and returns the time it took to obtain the N tokens
- TryAcquire () returns false if tryAcquire() cannot obtain the token
- TryAcquire (int permits) obtains a specified number of tokens, if not, return false immediately
- TryAcquire (long Timeout, TimeUnit Unit) Determines whether the token can be obtained within the specified time. If not, return false immediately
- TryAcquire (int permits, long timeout, TimeUnit unit
Step 3: Experience the effect
By accessing the test address: http://127.0.0.1:8080/limit/test1, refresh and observe the back-end log
WARN LimitController:35- Enter service downgrade time2021- 09 -25 21:39:37
WARN LimitController:35- Enter service downgrade time2021- 09 -25 21:39:37
INFO LimitController:39- Token acquisition success, time2021- 09 -25 21:39:37
WARN LimitController:35- Enter service downgrade time2021- 09 -25 21:39:37
WARN LimitController:35- Enter service downgrade time2021- 09 -25 21:39:37
INFO LimitController:39- Token acquisition success, time2021- 09 -25 21:39:37
WARN LimitController:35- Enter service downgrade time2021- 09 -25 21:39:38
INFO LimitController:39- Token acquisition success, time2021- 09 -25 21:39:38
WARN LimitController:35- Enter service downgrade time2021- 09 -25 21:39:38
INFO LimitController:39- Token acquisition success, time2021- 09 -25 21:39:38
Copy the codeAs can be seen from the above log, there are only two successful attempts within 1 second, and the others fail and degrade, indicating that we have successfully added the traffic limiting function to the interface.
Of course, we can’t use this directly in real development. As for the reason, think about it, you need to manually add tryAcquire() to each interface, mixing business code with limiting code, and obviously violating the DRY principle, code redundancy, duplication of effort. I’m gonna be laughed at by the old birds in code reviews. What the hell!
So, we need to find a way to optimize it here – with custom annotations +AOP interface limiting.
Implement interface flow limiting based on AOP
AOP based implementation is also very simple, the implementation process is as follows:
Step 1: Add AOP dependencies
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
Copy the codeStep 2: Customize stream limiting annotations
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD})
@Documented
public @interface Limit {
/** * Resource key, unique * function: different interface, different flow control */
String key(a) default "";
/** * Maximum number of access restrictions */
double permitsPerSecond (a) ;
/** * The maximum waiting time to get a token */
long timeout(a);
/** * Maximum waiting time to obtain a token, in minutes/seconds/ms. Default: ms */
TimeUnit timeunit(a) default TimeUnit.MILLISECONDS;
/** * no token prompt */
String msg(a) default"System busy, please try again later.";
}
Copy the codeStep 3: Use AOP facets to intercept limiting annotations
@Slf4j
@Aspect
@Component
public class LimitAop {
/** * Different interfaces, different traffic control * map key is Limiter. Key */
private final Map<String, RateLimiter> limitMap = Maps.newConcurrentMap();
@Around("@annotation(com.jianzh5.blog.limit.Limit)")
public Object around(ProceedingJoinPoint joinPoint) throws Throwable{
MethodSignature signature = (MethodSignature) joinPoint.getSignature();
Method method = signature.getMethod();
// Take the limit annotation
Limit limit = method.getAnnotation(Limit.class);
if(limit ! =null) {
// Key function: Different interfaces, different flow control
String key=limit.key();
RateLimiter rateLimiter = null;
// Verify that the cache has a key hit
if(! limitMap.containsKey(key)) {// Create a token bucket
rateLimiter = RateLimiter.create(limit.permitsPerSecond());
limitMap.put(key, rateLimiter);
log.info("New token bucket ={}, capacity ={}",key,limit.permitsPerSecond());
}
rateLimiter = limitMap.get(key);
/ / get the token
boolean acquire = rateLimiter.tryAcquire(limit.timeout(), limit.timeunit());
// If no command is obtained, an exception message is returned
if(! acquire) { log.debug("Token bucket ={}, failed to get token",key);
this.responseFail(limit.msg());
return null; }}return joinPoint.proceed();
}
/** * throws an exception directly to the front end@paramMSG Prompt message */
private void responseFail(String msg) { HttpServletResponse response=((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getResponse(); ResultData<Object> resultData = ResultData.fail(ReturnCode.LIMIT_ERROR.getCode(), msg); WebUtils.writeJson(response,resultData); }}Copy the codeStep 4: Annotate the interface that needs traffic limiting
@Slf4j
@RestController
@RequestMapping("/limit")
public class LimitController {
@GetMapping("/test2")
@ Limit (key = "limit2", permitsPerSecond = 1, the timeout = 500, timeunit timeunit = the MILLISECONDS, MSG = "the current line number is more, please try again later." )
public String limit2(a) {
log.info("Token bucket Limit2 Token obtaining succeeded");
return "ok";
}
@GetMapping("/test3")
@ Limit (key = "limit3", permitsPerSecond = 2, the timeout = 500, timeunit timeunit = the MILLISECONDS, MSG = "system is busy, please try again later." )
public String limit3(a) {
log.info("Token bucket Limit3 Token obtaining success");
return "ok"; }}Copy the codeStep 5: Experience the effect
By accessing the test address: http://127.0.0.1:8080/limit/test2, refresh and observe the output:
Normal response:
{"status":100."message":"Operation successful"."data":"ok"."timestamp":1632579377104}
Copy the codeTrigger current limiting:
{"status":2001."message":"System busy, please try again later!"."data":null."timestamp":1632579332177}
Copy the codeThrough observation, the effect of interface limiting is also achieved based on custom annotations.
summary
Generally, when the system is online, we can evaluate the performance threshold of the system through pressure measurement of the system, and then add reasonable current limiting parameters to the interface to prevent the direct crushing of the system when there is a large flow request. Today we introduced several common flow limiting algorithms (with emphasis on token bucket algorithm), based on Guava tool class to achieve interface flow limiting and use AOP to complete the optimization of the flow limiting code.
After the completion of optimization business code and flow limiting code decoupling, developers as long as a note, do not care about the flow limiting implementation logic, and reduce code redundancy greatly improve the code readability, code review who can laugh at you again?
Well, this is the end of today’s article. Finally, I am Miao Jam, an architect who writes codes and a programmer who does architecture. I look forward to your forwarding and attention.
Old bird series source code has been uploaded to GitHub, need in the public number [JAVA Daily records] reply keyword 0923 to obtain the source address.