1. What is Service Limit?
With the development of micro-service and distributed system, the mutual call between various services is more and more complex.
In order to ensure the stability and high availability of its own services, when faced with the request call beyond its own service capacity, it should do some flow limiting measures.
As the May Day, National Day period of travel, scenic spots full, tourists restricted flow. Our service also needs to do service limit in the face of scenarios with high concurrency and large traffic, such as seckill, big promotion, 618, Double 11, possible malicious attacks, crawlers and so on.
Concept: Intercepting requests that exceed the capacity of the service to handle, limiting the traffic that can access the service.
2. Commonly used current limiting algorithms
2.1 Counter method (fixed window)
Concept: The number of incoming requests is counted per unit time. When the statistics reach the threshold of flow limiting, flow limiting (such as rejection and queuing) is started. At the end of this unit time, the calculator clears and starts again.
Problem: At a fixed time window switch, up to 2 times the threshold traffic may be received.
If the time window is set to 5 seconds, the current limiting threshold is 10.
At the fifth second, 10 traffic came in, and the traffic in the window did not exceed the threshold. At the 6th second, switch to the next window. At this time, the incoming traffic is 10, and the current window traffic still does not exceed the threshold value. However, at the switching critical point between the 5th and 6th seconds, double the traffic of the current limiting threshold is entered in a short period of time, and the service may experience an exception due to excessive request traffic.
2.2 Sliding window algorithm
Concept: The sliding window refines and refines the fixed window by dividing a time window into several time panes. Each pane represents a fixed period of time (such as 1 minute) and has an independent counter. After a fixed time (such as 1 minute), the sliding window is moved forward one grid. The thinner the panes in the sliding window are, the more accurate the current limiting statistics are.
If the time window is set to 5 seconds, the current limiting threshold is 10.
At the fifth second, 10 traffic came in, and the traffic in the window did not exceed the threshold. At the 6th second, the incoming flow is 10. At the same time, the sliding window is moved to the right one grid. At this time, the flow in the sliding window (2nd to 6th second) is 20, which is greater than the current limiting threshold, and the current limiting begins.
2.3 Funnel algorithm
Concept: The request is poured into the funnel like water, and then flows out at a fixed rate. Requests can enter until the funnel is full. If the funnel is full, the request is rejected.
Funnel algorithm can smooth traffic, but it can't solve the problem of traffic surge.
2.4 Token Bucket Algorithm
Concept: A token is put into the token bucket at a constant rate (token generation rate), and cannot be placed when the token bucket is full (token bucket size). When the request arrives, it gets the token, then the request is processed and the obtained token is deleted. When the token is insufficient, the request cannot obtain the token and the request is rejected.
The Token Bucket algorithm smooths the flow limit while tolerating burst traffic.
3. Current limiting strategy
3.1 Denial of Service
When the request traffic reaches the current limiting threshold, the redundant requests are rejected directly. Can be designed to achieve the specified domain name, IP, client, application, users and other different sources of the request for rejection.
3.2 Delay processing
The excess requests are added to the cache queue or the delay queue to deal with the short-term traffic surge, and the accumulated request traffic is gradually processed after the peak.
3.3 Request grading (priority)
Prioritize requests from different sources and process higher priority requests first. Such as VIP customers, important business applications (such as transaction services have priority over logging services)
3.4 Dynamic current limiting
System related indicators can be monitored, system pressure can be evaluated, and current limiting threshold can be dynamically adjusted through registration center, configuration center, etc.
3.5 Monitoring and warning & dynamic capacity expansion
If there is an excellent service monitoring system and automatic deployment and release system, the system can automatically monitor the operation of the system through the monitoring system, and give early warning in the form of email, SMS and other methods in the case of sudden increase in service pressure and large volume of traffic writing in the short term.
Under certain conditions, relevant services can be automatically deployed and released to achieve the effect of dynamic capacity expansion.
4. Current limiting position
4.1 Access Layer Current Limitation
The domain name or IP can be restricted through NGINX, API routing gateway, etc., and illegal requests can be intercepted at the same time
4.2 Application of current restriction
Each service can have its own stand-alone or cluster traffic limiting measures, or it can call a third-party traffic limiting service
4.3 Basic service limitation
Database: Restrict database connections and read/write rates
Message Queuing: Limit consumption rate (consumption, consumption thread)
If this article is useful to you, click on it
praiseGo bai!
If you have any questions, please leave a comment!
Welcome to reprint, please indicate the source!