What is a current restrictor?
A current limiter is a mechanism that limits the number of times an operation can be performed in a given period of time (for example, 5 times per second) or the number of times it can be performed (for example, 1 gigabyte of data per second).
Flow limiter is a kind of defensive programming implementation. In the case of large data volume and high concurrent access, services or interfaces are often unavailable in the face of skyrocketing requests, or even lead to chain reaction and the whole system crash. One of the techniques you need to use at this point is limiting traffic, waiting, queuing, demoting, denial of service, etc., when a request reaches a certain number of concurrent requests or rates.
Why do we need a current limiter
In a large distributed system, there are many aspects to consider in system design:
- Dynamic system expansion and shrinkage, there will always be a lag. There will always be peaks and valleys in business. The cluster size does not always run at its peak size, which is too expensive and usually has a dynamic expansion strategy. But this kind of dynamic expansion, there is usually a lag, can not ensure that the instantaneous high flow is handled well. A traffic limiter ensures that the incoming traffic of a certain service does not cause other services to fail.
- Cascading failure (cascading failure) : distributed systems tend to have a health check, also tend to have break down mechanism, flow peak, when a node overload, cause the failure of the node health check offline, or open circuit breaker, causing the node flow into the other nodes also overload lead to other nodes.
- For a public service, traffic limiting is required for different tenants or users to prevent a user from stealing all resources.
- Flow control: To prevent high load on one node while low load on other nodes. In addition to load balancing control, a current limiter is also required to ensure that the pressure of a node is not too high.
Take a simple example: imagine a mall that has two businesses: placing orders and viewing your own orders. In a limited second kill, user orders suddenly spike at a certain point. The system may not have the capacity to handle such a large number of concurrent orders, resulting in requests being blocked, queued, and, in turn, all resources being consumed by orders, and users’ requests to view their orders being unable or slow to execute. At the same time, users will continue to brush requests, resulting in further requests piling up.
Design of relevant strategies for current limiter
If a current limiter is not used at all, it is generally necessary to prevent overloading by setting appropriate request timeouts, minimizing synchronous wait queues, and appropriate disconnection policies. However, this approach does not avoid the four problems mentioned above.
In the current microservices architecture, a process is generally both a service provider and a service caller. This is especially true under the service grid. For service providers, traffic limiting is to control external traffic to prevent excessive pressure. For service call traffic limiting, the main consideration is pressure evenness (although the service call generally has load balancing algorithm, but the general load balancing algorithm can not guarantee the true load balance, the client flow limiter can further help to prevent all the pressure to a certain instance).
For server traffic limiting, when traffic limiting is triggered, the server generally rejects the request and may return the HTTP status code 429. Depending on the client’s policy, the client can either simply exception the request or cache it and retry it.