In the nginx traffic limiting configuration article, we introduced how to use nginx traffic limiting. This article introduces another traffic limiting method, RateLimiter.
V Current limiting background
In the early computer field, time limiting technology is used to control the speed at which network interfaces can send and receive communication data. It can be used to optimize performance, reduce latency and improve bandwidth. Now in the field of the Internet, also draw on this concept, used for service control request rate, if double eleven limit the flow, 12306 grab tickets and so on. Even in fine-grained software architectures, similar concepts exist.
When the system uses downstream resources, it needs to consider the resource limitation and processing capability of the downstream. If the downstream resources cannot improve the processing performance in a short period of time, you can use a traffic limiter or similar protection mechanism to prevent the overall service unavailability caused by the collapse of downstream services.
V Current limiting algorithm
There are two common traffic limiting algorithms: leaky bucket algorithm and token bucket algorithm.
2.1 Leaky bucket algorithm
The Leaky Bucket algorithm is commonly used in Traffic Shaping or Rate Limiting on the network. It controls the Rate at which data is injected into the network and smoothen burst Traffic on the network. The leaky bucket algorithm provides a mechanism by which burst traffic can be shaped to provide a steady flow of traffic to the network.
A leaky bucket can be thought of as a single-server queue with a constant service time, and if the leaky bucket (packet cache) overflows, packets are discarded. In a network, the leak-bucket algorithm can control the port traffic output rate, smooth the burst traffic on the network, and implement traffic shaping to provide a stable traffic for the network.
As shown in the figure below, the request is likened to water. When the water comes, it is put into a bucket first and discharged at a limited speed. When the water comes too fast but not fast enough, the water will overflow directly, that is, denial of service.
Pictures from the network, deleted.
It can be seen that the leaky bucket algorithm can control the access speed of traffic well. Once the speed exceeds the speed, services are denied.
2.2 Token bucket algorithm
Token bucket algorithm is one of the most commonly used algorithms in Traffic Shaping and Rate Limiting. Typically, the token bucket algorithm is used to control the amount of data sent to the network and to allow the delivery of burst data.
The token bucket algorithm works by putting tokens into the bucket at a constant rate. If the request needs to be processed, a token needs to be fetched from the bucket first. When no token is available in the bucket, the service is denied. In principle, the token bucket algorithm and the leaky bucket algorithm are opposite, one is “water”, the other is “water”.
In addition to limiting the average data transfer rate, some degree of burst transmission is required for many application scenarios. In this case, the leaky bucket algorithm may not be suitable, but the token bucket algorithm is more suitable.
As shown in the figure below, the token bucket algorithm works by putting tokens into the bucket at a constant rate. If the request needs to be processed, a token needs to be obtained from the bucket first. When no token is available in the bucket, the service is denied.
Pictures from the network, deleted.
RateLimiter(Google’s Guava package), described in an example later in this article, is the token bucket algorithm used.
2.3 Difference between leaky bucket algorithm and token bucket algorithm
The flow rate of the leaky bucket algorithm is constant, which means that if there is a large instantaneous flow, most of the requests will be discarded (also known as overflow). Leaky bucket algorithms can often be used to limit traffic to external interfaces to protect other systems, such as when we request a bank interface, and usually limit the number of concurrent requests.
The token bucket algorithm generates tokens at a constant rate, while there is no speed limit for requesting tokens. This means that in the face of instantaneous heavy traffic, the algorithm can request a large number of tokens in a short period of time, can handle the instantaneous traffic, and the process of token acquisition is not a very expensive thing. Token bucket algorithms can often be used to limit access traffic and protect their own systems.
Note that in some cases, leaky bucket algorithms cannot efficiently use network resources because the leaky bucket leakage rate is fixed. Therefore, leaky bucket algorithms cannot make a single data flow reach the port rate even if there is no congestion in the network. Therefore, leaky bucket algorithm is inefficient for traffic with burst characteristics. The token bucket algorithm can satisfy the traffic with burst characteristics. In general, the leaky bucket algorithm is combined with the token bucket algorithm to provide more efficient control of network traffic.
vRateLimiter
3.1 Basic Introduction
RateLimiter conceptually, a RateLimiter allocates licenses at configurable rates. If necessary, each acquire() blocks the current thread until the license is available. Once a permit has been obtained, it does not need to be released.
RateLimiter uses a flow control algorithm called a token bucket. RateLimiter throws tokens into the bucket at a certain frequency until the thread gets the token. For example, if you want your application’s QPS to be less than 1000, RateLimiter sets the rate to 1000. I’m going to throw 1,000 tokens into the bucket every second.
3.2 Method Summary
| Modifiers and types | Methods and Description |
|---|---|
| double | Acquire () acquires a license from RateLimiter, which blocks until the request is obtained |
| double | Acquire (int permits) obtains the specified number of permits from RateLimiter, which is blocked until the request is obtained |
| static RateLimiter | Create (double permitsPerSecond) Creates RateLimiter based on the specified steady throughput rate, where the throughput rate is the number of permissions per second (usually QPS, queries per second) |
| static RateLimiter | Create (double permitsPerSecond, long warmupPeriod, TimeUnit unit) creates RateLimiter based on the specified steady throughput rate and warmupPeriod, where the throughput rate is the number of permitsPerSecond (usually QPS, Requests per second), during this warmup period, the number of permits allocated per second by RateLimiter increases steadily until it reaches its maximum rate at the end of the warmup period. (As long as there are enough requests to saturate it) |
| double | GetRate () returns the steady rate in the RateLimiter configuration, in permits per second |
| void | SetRate (double permitsPerSecond) The steady rate at which the RateLimite is updated. PermitsPerSecond is provided by the factory method that constructed the RateLimiter. |
| String | ToString () returns the character representation of the object |
| boolean | TryAcquire () obtains permission from RateLimiter if the permission can be obtained immediately without delay |
| boolean | TryAcquire (int permits) obtain permits from RateLimiter, if the permits can be obtained immediately with no delay |
| boolean | TryAcquire (int permits, long Timeout, TimeUnit Unit) obtains the specified permitage from RateLimiter if the permitage can be obtained within the timeout period, Or if you cannot get the number of permissions before timeout expires, return false immediately (without waiting) |
| boolean | TryAcquire (Long Timeout, TimeUnit Unit) obtain permission from RateLimiter if the permission can be obtained within the timeout period, or if the permission cannot be obtained before timeout expires, Return false immediately (no waiting) |
3.3 the
3.3.1 Adding a Reference
<dependency> <groupId>com.google.guava</groupId> <artifactId> <version>20.0</version> </dependency> < the dependency > < groupId > org. Aspectj < / groupId > < artifactId > aspectjweaver < / artifactId > < version > 1.8.10 < / version > </dependency>Copy the code3.3.2 Adding annotations
package learn.web.interceptor; import java.lang.annotation.*; */ @target (ElementType.METHOD) @Retention(retentionPolicy.runtime) Public @interface Limiting {// Default token double limitNum() default 20; String name() default ""; }Copy the code3.3.3 aop aspects
package learn.web.interceptor; import com.google.common.util.concurrent.RateLimiter; import lombok.extern.slf4j.Slf4j; import org.aspectj.lang.ProceedingJoinPoint; import org.aspectj.lang.Signature; import org.aspectj.lang.annotation.Around; import org.aspectj.lang.annotation.Aspect; import org.aspectj.lang.annotation.Pointcut; import org.aspectj.lang.reflect.MethodSignature; import org.springframework.stereotype.Component; import java.lang.reflect.Method; import java.util.concurrent.ConcurrentHashMap; /** * @author toutou * @date by 2020/12 * @des */ @Aspect @Component @Slf4j public class RateLimitAspect { private ConcurrentHashMap<String, RateLimiter> RATE_LIMITER = new ConcurrentHashMap<>(); private RateLimiter rateLimiter; @Pointcut("@annotation(learn.web.interceptor.Limiting)") public void serviceLimit() { } @Around("serviceLimit()") public Object around(ProceedingJoinPoint Point) throws Throwable {// Obtain the intercepting method name. Signature sig = point.getSignature(); MethodSignature msig = (MethodSignature) sig; Object target = point.gettarget (); CurrentMethod = target.getClass().getMethod(msig.getName(), msig.getTypes ()); . / / annotation information Limiting the annotation = currentMethod getAnnotation (Limiting. Class); double limitNum = annotation.limitNum(); // Get annotations added to the bucket per second token String functionName = msig.getName(); If (RATE_LIMITER. ContainsKey (functionName)){rateLimiter=RATE_LIMITER. Get (functionName); }else { RATE_LIMITER.put(functionName, RateLimiter.create(limitNum)); rateLimiter=RATE_LIMITER.get(functionName); } if(ratelimite.tryacquire ()) {log.info(" processing done "); return point.proceed(); } else {throw new RuntimeException(" Server busy, please try again later." ); }}}Copy the code3.3.4 Adding a Controller Test
package learn.web.controller; import learn.model.vo.Result; import learn.web.interceptor.Limiting; import lombok.extern.slf4j.Slf4j; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; /** * @author toutou * @date by 2020/12 * @des */ @slf4j @restController Public class IndexController {/** ** * @return */ @GetMapping("/limit1") @Limiting(limitNum = 1, name = "limiting1") public Result Limit1() { return Result.setSuccessResult("limiting1"); } /** * interface 2 * @return */ @getMapping ("/limit2") @limiting (limitNum = 5, name = "limiting2") public Result Limit2() { return Result.setSuccessResult("limiting2"); }}Copy the code3.3.5 Test effect
Ab Test Screenshot
Browser Screenshot
Other reference/learning materials:
- RateLimiter (Guava: Google Core Libraries for Java HEAD-jre-SNAPSHOT API)
- Quick Guide to the Guava RateLimiter
- High concurrency system flow limiting – leakage bucket algorithm and token bucket algorithm
- ratelimiter/
- Interface traffic limiting algorithm
- RateLimiter current-limiting
V Source code address
Github.com/toutouge/ja…
About the author: Focus on basic platform project development. If you have any questions or suggestions, please feel free to comment! Copyright notice: The copyright of this article belongs to the author and the blog garden, welcome to reprint, but without the consent of the author must retain this statement, and give the original text link in a prominent place on the page of the article. For the record: all comments and messages will be answered as soon as possible. You are welcome to correct your mistakes and make progress together. Or direct private message I support the blogger: if you think the article is helpful to you, you can click on the lower right corner of the article [recommendation]. Your encouragement is the author to adhere to the original and continuous writing of the biggest power! \